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EXECUTIVE  SUMMARY 

During  times  of  peace,  health  care  is  one  of  the  foremost  quality  of  life  issues  to  active 
duty  members,  their  families  and  retirees.  Patient  satisfaction  surveys  are  used  to 
determine  how  patients  perceive  salient  aspects  of  their  medical  care.  There  has  been 
substantial  anecdotal  evidence  to  suggest  that  patients  are  unhappy  with  their  care,  but 
past  analysis  of  the  DOD  Annual  Surveys  using  simple  frequencies  of  responses  indicated 
that,  overall,  patients  were  satisfied.  This  thesis,  using  a  powerful  new  technique  called 
Trellis  Graphics  that  allows  more  than  three  variables  to  be  visualized  simultaneously, 
has  uncovered  startling  results  that  go  beyond  previous  analysis,  provide  evidence  to 
support  the  anecdotal  claims,  and  show  that  overall  satisfaction  is  not  a  reliable 
measurement  for  determining  patient  satisfaction.  The  seven  factors  defined  by  the 
National  Committee  on  Quality  Assurance  are  each  individually,  and  together  as  a  group, 
more  reliable  measures.  The  inability  to  choose  a  provider  was  clearly  rated  by  every 
beneficiary  group  as  the  single  greatest  source  of  dissatisfaction.  There  are  also 
differences  in  satisfaction  between  the  sexes,  and  among  the  different  groups.  Active 
duty  members,  who  are  the  primary  customers  of  military  treatment  facilities,  are  the 
most  dissatisfied,  and  women  tend  to  be  less  happy  than  men. 

This  thesis  provides  a  new  technique  for  analysis  of  complex  surveys  and  attitudinal 
studies  that  could  have  far-reaching  impact  on  how  surveys  are  analyzed  throughout  the 
Department  of  Defense.  Surveys  are  the  central  tool  used  within  DOD  for  measuring 
intangible  factors  such  as  attitudes,  feelings,  and  quality  of  life  issues.       Almost  all 
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surveys  in  DOD  use  Likert  scales  that  is,  they  collect  discrete  ordinal  responses.  The 
accepted  analysis  of  an  ordinal  scale  survey  is  to  use  simplistic  frequency  tables  or  to  use 
logistic  regression  for  more  advanced  work.  Logistic  regression  for  the  patient 
satisfaction  data  is  unsatisfactory  because  of  the  huge  models  that  resulted  from  the 
complex  interactions  among  the  demographic  and  the  measurement  variables.  At  this 
point,  after  the  accepted  types  of  analysis  were  used  and  failed,  a  relatively  new  graphing 
technique  called  Trellis  Graphics  was  adapted  to  understand  the  complex  interactions  that 
were  occurring  within  the  data.  This  technique  allows  the  user  to  look  past  the  three- 
dimensional  barrier  and  visualize  the  hypervariate  data  in  four  and  five  dimensions.  This 
graphics  technique  had  previously  been  used  to  analyze  agriculture  and  environmental 
measures,  not  attitudinal  surveys.  The  results  were  both  clear  and  useful.  One  important 
impact  of  the  research  found  an  area  of  major  dissatisfaction  among  active  duty  members 
and  discovered  a  possible  flaw  in  the  DOD  survey  design.  The  most  significant  impact  of 
this  research  is  putting  into  practice  a  new  method  that  introduces  a  fundamentally  new 
approach  to  analysis  of  survey  results,  never  before  considered.  Both  the  analyst  and  the 
end  user  can  visualize  the  data,  which  adds  confidence  to  the  decision-maker  and 
enhances  the  overall  benefit  of  the  analysis.  Trellis  graphics  could  be  used  by  all  of  the 
military  services  to  analyze  and  amplify  the  underlying  foundations  of  attitudes  and 
feelings  for  many  of  the  quality  of  life  issues  such  as  retention,  housing,  pay,  and  medical 
care.  It  is  these  issues  that  DOD  and  the  Congress  have  realized  are  vitally  important  to 
preserving  today's  peacetime  military  force  at  its  highest  levels  of  readiness.  The 
technique  introduced  herein,  can  make  a  significant  contribution  to  the  understanding  of 


those  issues  and  ultimately  enhance  decision  maker's  ability  to  improve  the  quality  of  life 
for  all  members  of  the  military  community. 


XI 


Xll 


I.  INTRODUCTION 

The  Department  of  Defense  operates  one  of  the  nation's  largest  healthcare  complexes, 
with  nearly  8.3  million  individuals  eligible  to  receive  care  through  the  Military  Health 
System.  Since  the  end  of  the  Cold  War,  the  U.S.  military  has  seen  manpower  reductions, 
which  have  changed  the  role  of  military  medicine.  Since  the  Department  of  Defense 
began  its  downsizing,  the  number  of  doctors,  nurses  and  medical  technicians  has  declined 
by  as  much  as  50  percent  in  some  locations.  Furthermore,  the  Base  Realignment  and 
Closure  Commission  (BRAC)  recommended  closing  a  number  of  installations  that  are  no 
longer  needed.  As  a  result  of  this  and  other  downsizing  efforts,  35  percent  of  the  military 
hospitals  that  existed  in  the  U.S.  in  1997  have  closed.  However,  the  total  number  of 
people  seeking  healthcare  through  the  MHS  has  dropped  by  only  9  percent.  (OSD  1997) 

In  order  to  mitigate  the  possible  decline  in  the  ability  to  care  for  its  patients,  the 
Military  Health  System  (MHS)  must  examine  itself  in  this  environment  and  move  toward 
the  standards  set  by  the  civilian  healthcare  industry.  These  standards  are  outlined  by  two 
of  the  most  prominent  healthcare  accrediting  bodies:  the  Joint  Commission  of 
Accreditation  of  Health  Care  Organization  (JCAHO)  and  the  National  Committee  on 
Quality  Assurance  (NCQA).  One  of  the  most  important  standards  detailed  by  these 
accrediting  organizations  and  the  focus  of  this  paper  is  the  measurement  of  a  patient's 
satisfaction.  In  the  future  when  all  hospitals,  military  and  civilian,  have  virtually  the 
same  kind  of  healthcare  products,  the  distinguishing  factor  that  will  ensure  an 
organization's  survival  will  be  its  attention  to  high-value  customer  services  (Nelson 
1991).  Therefore,  in  addition  to  high-quality  healthcare  products,  patient  satisfaction  will 
be  the  key  ingredient  for  an  organization  to  be  competitive  (Nelson  1991). 


In  a  recent  study  for  the  White  House  office  of  Consumer  Affairs  it  was  claimed: 

•  "96  percent  of  unhappy  customers  never  complain  about  rude  or  discourteous 
treatment,  but 

•  90  percent  or  more  who  are  dissatisfied  with  the  service  they  receive  will  not  buy 
again  or  come  back.  Worse  still, 

•  Each  of  those  unhappy  customers  will  tell  his  or  her  story  to  at  least  nine  other 
people,  and 

•  13  percent  of  those  unhappy  former  customers  will  tell  their  stories  to  more  than 
twenty  people  "  (Desatnick  1987) 


The  U.S.  healthcare  industry  in  general,  including  the  MHS,  is  undergoing  sweeping 
changes.  The  healthcare  industry  is  experiencing  a  shift  in  focus  from  specialty  providers 
to  medical  generalists,  from  inpatient  to  outpatient  care,  and  from  traditional  indemnity 
insurance  to  prepaid  health  plans.  As  the  Military  Health  System  moves  towards  the 
future,  its  leaders  have  many  challenges  facing  them.  Ensuring  customers  are  satisfied 
with  the  products  from  both  the  military  hospitals  and  the  network  of  civilian  providers 
will  be  the  key  to  providing  care  for  beneficiaries. 
A.  PURPOSE  AND  STRUCTURE  OF  THESIS 

Patient  satisfaction  has  been  shown  to  be  positively  correlated  to  treatment 
compliance,  decreasing  no-show  appointments,  better  outcomes,  and  decreased 
disenrollment  from  health  plans  (Cleary  1989).  The  primary  objective  of  this  paper  is  to 
determine  if  there  are  key  areas  such  as:  1)  access  to  healthcare,  2)  choice  of  providers 
and  continuity  of  care,  3)  perceptions  of  technical  competence  of  providers,  4)  overall 
quality  of  healthcare,  5)  communication  between  providers  and  patients,  6)  perception  of 
being  able  to  get  help  with  financial  needs,  and  7)  interpersonal  skills  of  the  providers, 
that  may  help  military  medical  facilities  improve  patient  satisfaction.  A  second  area  of 
interest  is  whether  there  are  distinct  differences  in  satisfaction  between  men  and  women 


in  the  different  regions.  Finally,  there  is  interest  in  whether  there  are  differences  in 
satisfaction  among  the  four  military  beneficiary  groups:  1)  active  duty  personnel,  2) 
family  members  of  active  duty  personnel,  3)  retirees,  their  family  members,  and  survivors 
under  age  65,  and  4)  retirees,  their  family  members,  and  survivors,  age  65  and  over. 

Access  was  of  primary  importance  to  each  of  the  four  beneficiary  groups  in  the  1994- 
95  Annual  Survey.  In  addition  to  access,  having  the  freedom  to  choose  one's  doctor  and 
location  of  care  may  be  of  great  importance  to  military  beneficiaries.  As  outlined  in 
chapter  55  of  Title  10  United  States  Code,  the  eligibility  status  of  the  individual  plays  a 
role  in  determining  the  location  of  care  he  or  she  may  obtain  from  either  the  civilian 
sector  or  a  military  facility,  thereby  affecting  his  or  her  choice  of  provider. 

This  thesis  will  begin  by  providing  some  examples  of  how  other  organizations  have 
been  using  customer  patient  satisfaction  surveys.  Chapter  II  gives  an  introduction  to  the 
Military  Health  Systems'  benefit  plan  and  provides  an  overview  of  the  types  of  medical 
plans  members  may  choose.  Then  Chapter  HI  will  give  a  brief  description  of  earlier  work 
on  patient  satisfaction,  discuss  a  working  definition  of  patient  satisfaction  and,  of  special 
importance,  present  an  overview  on  how  the  National  Committee  for  Quality  Assurance 
(NCQA)  has  defined  patient  satisfaction.  Chapter  IV  will  describe  the  1996  Health  Care 
Survey  of  DoD  Beneficiaries  Form  A  and  the  analysis  of  the  survey.  Chapter  V  will 
describe  the  results  of  logistic  regression  and  the  relatively  new  visualization  technique  of 
Trellis  Graphics  using  a  subset  of  the  1996  survey  data.  And  lastly,  Chapter  VI  discusses 
the  implications  of  the  analysis  and  gives  conclusions  and  recommendations. 


B    CUSTOMER  SURVEYS 

Customer  surveys  have  been  used  as  a  primary  tool  in  both  the  military  and  civilian 
markets  for  assessing  whether  patients  are  satisfied  with  the  different  aspects  of  their 
medical  care.  In  the  past,  patient  satisfaction  surveys  were  primarily  used  by  hospitals 
and  managed  care  organizations  as  marketing  tools  (Kennedy  1996).  Many 
organizations  were  comfortable  with  a  fairly  low  response  rate,  while  some  organizations 
even  appeared  to  believe  that  simply  giving  a  patient  a  survey  indicated  that  the 
organization  cared  about  the  patient's  opinion,  regardless  of  whether  or  not  the  survey 
results  were  actually  used  (Kennedy  1996).  In  today's  competitive  environment  and  with 
healthcare  costs  rising,  hospitals  and  managed  care  organizations  are  now  realizing  they 
need  more  reliable  and  useful  data  from  patient  satisfaction  surveys. 

Four  primary  conditions  have  prompted  the  change  in  this  direction  and  redesign  of 
surveys.  First,  and  foremost,  the  Joint  Commission  of  Accreditation  of  Hospital 
Organizations  and  the  National  Committee  on  Quality  Assurance  have  released  standards 
requiring  organizations  to  continuously  measure  patient  satisfaction.  These  measures  are 
to  provide  data  and  feedback  to  both  the  consumers  and  purchasers  of  healthcare.  The 
second  condition  is  that  healthcare  is  becoming  increasingly  competitive  and  costly 
which  leads  to  a  heavy  emphasis  on  the  resulting  quality  of  healthcare  services.  Thirdly, 
the  patient's  viewpoint  has  become  much  more  important  and  relevant  in  the  adoption  of 
continuous  quality  improvement  methods  and  programs.  Lastly,  employers  as  purchasers 
of  healthcare  are  demanding  information  on  satisfaction,  costs,  doctor/patient  relations, 
convenience  and  accessibility  for  their  employees.  (Kennedy  1996) 


Several  prominent  organizations  have  used  patient  satisfaction  surveys  successfully 
and  have  been  able  to  link  customer  judgments  with  process  measures.  One  example  of 
this  is  the  Henry  Ford  Medical  Group,  one  of  the  largest  group  medical  practices  in  the 
United  States  that  specializes  in  primary  care,  located  in  Detroit,  Michigan.  This 
organization  was  able  to  use  a  survey  to  identify  problems  that  their  customers  had  with 
access.  Two  of  the  major  problems  identified  were  that  patients  could  not  get  through  on 
the  telephone  to  obtain  one  of  the  limited  number  of  appointments,  and  that  they  had  to 
wait  for  long  periods  of  time  between  scheduling  an  appointment  and  seeing  the  doctor. 
The  Group  used  a  survey  to  ask  their  patients  about  key  issues  that  influenced  their 
overall  satisfaction.  The  survey  results  showed  that  the  significant  determinants  for 
satisfying  their  patients,  were  ease  of  scheduling  appointments,  being  able  to  see  a 
specialist  when  needed,  having  a  shorter  waiting  time  to  see  a  doctor,  and  the  ability  to 
obtain  medical  advice  via  phone.  (Anctil  1996) 

In  another  study,  a  survey  was  conducted  on  inpatient  care  at  Brigham  and  Women's 
Hospital  in  Boston.  The  intent  of  the  study  was  to  seek  general  information  about  patient 
satisfaction,  and  to  investigate  relationships  between  overall  satisfaction  and 
characteristics  such  as  age,  gender,  health  status,  source  of  payment  and  location  in  the 
hospital  where  the  patients'  received  care.  The  Brigham  and  Women's  Hospital  found 
that  for  inpatient  care,  perceived  health  and  their  satisfaction  with  nurses  and  physicians 
were  the  two  strongest  indicators  of  patient  satisfaction.  (Cleary  1989) 


H.  MILITARY  HEALTH  SYSTEM 
A.  CURRENT  HEALTH  BENEFITS  PLAN 

The  Military  Health  Services'  primary  mission  is  to  maintain  the  health  of  its  active 
duty  service  personnel  and  to  be  prepared  to  deliver  health  care  during  times  of  war.  In 
addition  to  its  primary  mission,  the  MHS,  as  an  employer,  offers  health  care  services  to 
non-active  duty  beneficiaries.  Title  10,  U.S.C.  Chapter  55,  Sections  1071-1106  outline 
the  entitlement  programs  for  the  Uniformed  Services  and  establishes  the  eligibility 
requirements  for  the  MHS.  The  healthcare  services  are  provided  through  a  system  of 
medical  centers,  smaller  hospitals,  and  clinics  worldwide,  and  through  a  DoD- 
administered  insurance-like  program  called  the  Civilian  Health  and  Medical  Program  of 
the  Uniformed  Services  (CHAMPUS).  CHAMPUS  is  automatically  available  to  families 
of  active  duty  personnel,  retirees  and  their  dependents  and  survivors  under  the  age  of  65. 
At  age  65,  beneficiaries  are  no  longer  eligible  for  CHAMPUS  because  they  become 
eligible  for  Medicare.    (TSO  1997) 

In  the  1980's,  MHS  faced  CHAMPUS  costs  which  grew  350  percent  while  the 
national  health  expenditures  increased  by  only  166  percent  during  the  same  period.  In 
response  to  the  escalating  costs,  DoD,  with  congressional  authorization,  has  been  testing 
alternative  approaches  for  delivering  healthcare  such  as  managed  care  which  emphasizes 
keeping  the  patient  healthy  through  preventive  care  and  services  rather  than  simply 
treating  the  patient  when  he  or  she  becomes  ill.  (GAO  1995) 

The  purpose  of  converting  the  MHS  to  a  managed  care  system  was  to  provide  quality 
care  while  improving  access  and  containing  costs.  This  initiative  toward  managed  care  in 
the  MHS  evolved  over  several  years.   The  Fiscal  Year  1988  Defense  Authorization  Act 


(Public  Law  100-180)  initiated  this  effort  by  mandating  demonstration  projects  be 
implemented  to  improve  quality  and  contain  costs  in  the  MHS.  These  projects  included 
the  CHAMPUS  Reform  Initiative  in  Hawaii  and  California  (CRI),  Catchment  Area 
Management  Demonstration  Projects,  and  the  U.S.  Army's  "Gateway  to  Care"  program 
implemented  in  1991.  All  of  these  initiatives  were  developed  to  contain  the  rapidly 
rising  healthcare  costs  for  CHAMPUS,  to  maintain  or  improve  the  quality  and 
accessibility  of  patient  care,  and  to  improve  both  beneficiary  and  Military  Treatment 
Facility  (MTF)  staff  satisfaction.  However,  under  some  of  the  demonstration  projects, 
costs  actually  increased:  in  the  case  of  CRI,  for  example,  by  4.6  percent  from  1988  to 
1989.  The  Assistant  Secretary  of  Defense  for  Health  Affairs  addressed  the  demonstration 
projects'  shortcomings  by  developing  TRICARE.  (GAO  1995) 
B.  TRICARE 

TRICARE  itself  is  a  combination  of  these  different  managed  care  projects.  It  was 
established  by  the  Secretary  of  Defense  under  the  authority  of  chapter  55  of  Title  10, 
United  States  Code,  principally,  section  1097.  The  military  health  system's  product  is  a 
benefits  package  modeled  after  the  civilian  sectors'  managed  care  plans,  preferred 
provider  plans  and  fee-for-service  plans.  TRICARE  is  a  regionally  managed  healthcare 
program  for  active  duty  and  retired  members  of  the  uniformed  services,  their  families  and 
survivors.  It  brings  together  the  healthcare  resources  of  the  Army,  Air  Force,  and  the 
Navy,  and  supplements  them  with  networks  of  civilian  healthcare  professionals  to 
provide  access  and  quality  services  while  maintaining  the  capability  to  support  military 
operations.  (OSD  1997) 


TRICARE  features  a  triple-option  benefit,  offering  beneficiaries  eligible  for 
CHAMPUS  two  new  options  for  health  care  in  addition  to  the  CHAMPUS  program.  The 
options  vary  in  the  choices  beneficiaries  have  in  selecting  their  physicians  and  the 
amount  beneficiaries  are  required  to  contribute  toward  the  portion  of  the  cost  of  their  care 
received  from  civilian  providers.  (OSD  1997) 

1.  TRICARE  Prime 

The  first  new  option,  the  TRICARE  Prime  option,  is  a  HMO-like  alternative  that 
provides  comprehensive  medical  care  to  beneficiaries  through  an  integrated  network  of 
military  and  contracted  civilian  providers.  Active  duty  members  are  automatically 
enrolled  in  this  option  and  beneficiaries  who  select  this  option  must  enroll  annually  in  the 
program,  agreeing  to  go  through  an  assigned  military  or  civilian  primary  care  physician 
for  all  of  their  care.  The  incentives  for  enrolling  in  Prime  are:  1)  no  enrollment  fees  for 
active  duty  and  their  dependents,  and  low  enrollment  fees  for  all  others,  2)  low  co- 
payments,  3)  no  annual  deductible,  and  4)  no  paperwork.  Retirees,  their  dependents,  and 
survivors  age  65  and  over  are  not  eligible  to  enroll  in  Prime  but  remain  eligible  to  receive 
care  in  military  medical  facilities.  TRICARE  prime  is  similar  to  health  maintenance 
organization  (HMO)  programs  offered  by  employers  across  the  U.S.  (OSD  1997) 

2.  TRICARE  Standard 

TRICARE  Standard  is  the  current  fee-for-service  CHAMPUS  program.  This  option 
provides  beneficiaries  the  greatest  freedom  of  choice  in  selecting  civilian  providers,  but  it 
also  requires  the  highest  beneficiary  costs  and  is  the  most  expensive  way  to  receive 
healthcare  in  the  MHS.  Beneficiaries  who  use  Standard  must:  1)  meet  an  annual 
deductible,  2)  pay  a  co-payment  of  20  percent  for  dependents  of  active  duty  and  25 


percent  for  retirees,  their  dependents,  and  survivors  of  active  duty  members,  and  3)  be 
responsible  for  CHAMPUS  claims  in  some  cases.  There  is  no  requirement  to  enroll  in 
the  Standard  option,  and  the  patient  may  still  seek  care  at  a  military  hospital  or  clinic  on  a 
space-available  basis.  (OSD  1997) 

3.  TRICARE  Extra 

TRICARE  Extra  is  the  second  new  option  and  lies  between  TRICARE  Prime  and 
Standard.  It  is  a  "preferred  provider  plan,"  through  which  beneficiaries  receive  a  5 
percent  discount  on  their  co-payments  if  they  use  the  civilian  contracted  network; 
however,  they  still  must  first  pay  their  deductible.  In  most  cases,  healthcare  will  be 
more  expensive  than  with  TRICARE  Prime  and  less  expensive  than  Standard,  but  the 
beneficiary  will  still  maintain  a  moderate  degree  of  freedom  of  choice  of  provider.  As 
with  the  Standard  option,  enrollment  is  not  required.  Patients  in  this  option  may  seek 
care  at  a  military  hospital  or  clinic  where  space  is  available,  but  most  of  those 
appointments  will  be  reserved  for  TRICARE  Prime  enrollees.  (OSD  1997) 
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m.  LITERATURE  REVIEW 

Patient  satisfaction  means  different  things  to  different  people.  It  is  important  to 
understand  it  in  terms  of  behavioral  models  and  to  give  it  a  working  definition.  In  1983 
Gregory  Pascoe  from  the  University  of  California  did  an  extensive  literature  review  on 
the  behavioral  models  of  patient  satisfaction  in  primary  health  care  settings.  The 
behavioral  models  provide  insight  into  what  things  such  as  attitudes,  expectations  and 
beliefs,  make  up  patient  satisfaction,  while  a  working  definition  provides  a  practical  way 
to  define  it  so  it  can  be  measured. 
A.  BEHAVIORAL  MODELS 

In  the  past,  patient  satisfaction  has  been  the  focus  of  three  different  theories 
summarized  in  the  literature:  expectancy-value  theories,  fulfillment  theories  and 
discrepancy  theories.  The  expectancy-value  model,  developed  by  Linder-Pelz  in  1982, 
proposed  that  attitudes  that  influence  expectations  are  the  primary  indicators  of  patient 
satisfaction.  Some  examples  are  attitudes  about  the  health  care  system  in  general, 
attitudes  about  the  location  of  care  received,  and  attitudes  about  the  cost  of  treatment.  An 
alternative  theory  is  the  fulfillment  theory,  which  defines  satisfaction  as  a  function  of  the 
amount  of  care  and  attention  received  from  a  situation  regardless  of  how  much  one  feels 
they  should  receive  or  how  much  they  would  want  to  receive.  The  third  model  is  the 
discrepancy  theory,  which  includes  the  patient's  perception  of  what  is  expected  and 
valued,  and  which  defines  satisfaction  by  the  difference  between  actual  outcome  and  the 
patient's  ideal  of  what  the  outcome  should  be.  (Pascoe  1983) 

Although  each  of  these  models  describes  patient  satisfaction  to  some  extent,  each  has 
some  area  of  logical  weakness.  The  expectancy-value  model  developed  by  Linder-Pelz  is 
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closest  to  general  acceptance.  However,  it  is  limited  because  although  satisfaction  in 
general  has  to  do  with  expectations  influenced  by  attitudes,  satisfaction  by  actual  services 
received  could  be  influenced  more  by  the  patient's  reaction  to  the  actual  experience  than 
to  their  initial  expectations.  On  the  other  hand,  the  fulfillment  model  assumes  that 
objective  outcomes  alone  will  determine  patient  satisfaction  and  ignores  the 
psychological  aspects  of  defining  care  and  services  received.  Lastly,  the  major 
disadvantage  of  the  discrepancy  theory  model  is  that  it  does  not  take  into  account  the 
reverse  implication,  which  is  the  assumption  that  any  difference  between  outcomes  will 
result  in  dissatisfaction  even  if  the  actual  outcome  was  better  than  the  patient's  original 
expectation  of  the  outcome.  It  has  been  shown  that  each  of  these  models  that  attempted 
to  define  patient  satisfaction  has  some  logical  weakness  and  furthermore,  none  has  been 
proven  by  empirical  evidence.  Because  of  these  problems,  researchers  turned  to 
consumer  satisfaction  models  for  insight  in  defining  patient  satisfaction.  (Pascoe  1983) 

Researchers  have  mostly  agreed  that  consumer  satisfaction  is  multidimensional, 
where  overall  satisfaction  depends  on  the  customer  identifying  important  attributes  of  a 
product  or  service,  and  then  determining  if  most  of  the  attributes,  or  the  most  important 
attributes,  are  deemed  favorable.  The  three  major  models  used  in  customer  satisfaction 
are  the  contrast  model,  the  assimilation  model  and  the  assimilation-contrast  model,  all  of 
which  use  the  expectancy  approach.  The  contrast  model  is  based  on  the  magnification  of 
differences  perceived  between  expectation  and  performance  by  the  consumer,  while  the 
assimilation  model  goes  one  step  further.  In  this  second  model,  consumers  will  adjust 
their  perceptions  of  performance  to  match  their  expectations  by  assimilating  the 
difference,  and  lower  their  expectations.    The  hybrid  of  the  above  two  models  is  the 


12 


assimilation-contrast  model.  With  this  model  it  is  accepted  that  expectations  are  the 
customer's  standard  of  judgement  but  there  is  some  measurement  of  assimilation  or 
latitude  of  acceptance  if  the  customer's  exact  expectations  are  not  met.  All  three  of  these 
models  have  been  empirically  tested  and  supported.  (Pascoe  1983) 

A  combination  of  the  patient  satisfaction  models  and  the  consumer  satisfaction  models 
can  be  used  to  conceptualize  what  patient  satisfaction  is  and  what  role  it  plays  in 
healthcare.  Psychological  factors  play  a  role  in  how  patient  satisfaction  is  defined. 
Pascoe  defines  patient  satisfaction  as  "a  healthcare  recipient's  reaction  to  salient  aspects 
of  the  context,  process,  and  result  of  [his  or  her]  service  experience"  (1983).  Patient 
satisfaction  is  then  a  comparison  of  a  subjective  standard  to  the  actual  healthcare 
experience  of  the  service  received.  This  may  be  based  upon  a  subjective  ideal,  a  sense  of 
what  is  deserved,  an  average  of  past  experience  in  other  situations,  or  upon  some  learned, 
minimally  acceptable  level  of  service.  (Pascoe  1983) 

In  understanding  patient  satisfaction  the  behavioral  models  give  us  insight  into  the 
human  factors  of  attitude  and  expectations.   However,  patient  satisfaction,  in  order  to  be 
measured,  must  be  defined  by  a  working  definition.  Research  has  shown  that  this  can  be 
done  through  defining  patient  satisfaction  in  terms  of  several  factors. 
B.  A  WORKING  DEFINITION  OF  PATIENT  SATISFACTION 

Throughout  the  literature,  there  are  some  common  themes  of  factors  that  indicate 
patient  satisfaction.  These  are  accessibility,  provider-patient  interaction,  clinical 
outcome,  and  choice  of  provider.  (Pascoe  1983) 

Access  is  one  of  the  most  important  factors  in  determining  patient  satisfaction.  It  has 
many  dimensions  such  as  availability,  accommodation,  affordability,  and  acceptability 
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(Penchansky  1983).  In  Pascoe's  review  of  recent  studies,  he  found  that  all  of  the  studies 
with  one  exception  had  shown  a  significant  relationship  between  increased  access  and 
patient  satisfaction  (1983).  Dissatisfaction  was  highly  correlated  with  difficulty  in 
scheduling  appointments,  length  of  waiting  time  between  appointments,  increased 
distance  to  travel  to  obtain  care,  and  longer  delays  in  obtaining  services  (Pascoe  1983). 

The  factor  of  provider-patient  interaction  has  typically  included  technical  competence, 
interpersonal  skills  of  the  provider  and  the  length  of  time  the  visit  lasts.  The  perception 
of  a  provider's  competence  is  strongly  interrelated  with  the  provider's  interpersonal  skills 
and  communication  skills.  High  patient  satisfaction  has  been  indicated  by  a  provider's 
ability  to  provide  clear  and  adequate  explanations,  which  in  turn  increase  the  patient's 
perception  of  the  provider's  technical  competence.  (Pascoe  1983) 

A  third  factor,  clinical  outcome,  has  had  mixed  reviews  on  whether  it  is  a  valuable 
factor.  Clinical  outcomes  are  represented  by  a  change  in  a  patient's  health  status  that  is 
directly  attributable  to  the  treatment  received.  Most  satisfaction  surveys  do  not  account 
for  the  patient's  condition  prior  to  treatment.  In  general,  however,  the  severity  of  illness 
or  health  status  of  the  patient  will  play  a  role  in  how  the  patient  feels  about  the  care  he  or 
she  has  received.  A  study  by  Stewart  and  Wanklin  in  1978  found  no  relationship 
between  outcomes  and  patient  satisfaction,  but  other  studies  have  found  small  to  medium 
correlation  with  patient  satisfaction.  (Pascoe  1983) 

The  final  factor  is  choice  of  provider.  There  has  been  significant  evidence  to  indicate 
that  the  provider-patient  fit  is  very  important  for  a  patient's  overall  satisfaction.  For 
example,  in  a  study  cited  in  the  journal  Hospitals,  Kaiser  Permanente  of  Northern 
California  surveyed  10,000  beneficiaries  and  found  that  choice  in  choosing  one's  own 
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primary  care  manager  was  the  single  most  important  predictor  of  overall  satisfaction 
(Greene  1998).  In  a  study  by  Ross  in  1982,  it  was  found  that  a  patient  was  more  likely  to 
have  increased  satisfaction  if  the  patient's  and  the  physician's  sociodemographics 
matched.  Within  this  dimension  of  provider-patient  fit  is  also  continuity  of  care.  Having 
a  regular  source  of  care  and  seeing  the  same  provider  are  strongly  related  to  high  patient 
satisfaction  (Pascoe  1983). 

In  addition  to  behaviors  and  the  above-mentioned  factors,  two  socio-demographic 
categories,  sex  and  age,  are  often  used  in  studies  on  patient  satisfaction.  In  general,  it 
was  found  from  studies  in  the  late  1970's  that  older  patients  tend  to  report  higher  levels 
of  satisfaction  than  do  younger  patients  and  that  women  tend  to  be  more  satisfied  than 
men  (Pascoe  1983).  However,  there  have  been  other  studies  from  that  same  time  period 
that  have  shown  younger  patients  were  more  satisfied  than  older  patients  (Pascoe  1983). 
A  survey  conducted  in  1989  at  Brigham  and  Women's  Hospital  in  Boston  had  mixed 
results  on  whether  sex  and  age  are  significant  in  predicting  patient  satisfaction.  Each  of 
these  variables  may  play  a  role  in  determining  satisfaction.  However,  these  could  be 
somehow  related  to  the  organization,  the  severity  of  illness,  or  other  factors  (Pascoe 
1983). 
C.  HEDIS 

Research  seems  to  show  patient  satisfaction  information  does  have  an  important  role 
in  evaluating  primary  health  care.  Because  of  its  importance,  the  National  Committee  for 
Quality  Assurance  (NCQA),  an  organization  whose  goal  is  to  report  publicly  on  the 
quality  of  managed  care  plans,  developed  a  system  for  standardizing  the  measurement  of 
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salient  aspects  of  quality  care.  This  system  is  called  the  Health  Plan  Employer  Data  and 
Information  Set  (HEDIS). 

HEDIS  is  a  report  card  project  that  consists  of  a  set  of  standardized  performance 
measures  developed  by  the  Committee  on  Performance  Measurement  (CPM).  The 
committee  was  made  up  of  representatives  of  many  different  populations  of  constituents 
from  both  the  public  and  the  private  healthcare  sectors.  These  include  representatives  of 
Medicare,  Medicaid,  organized  labor,  medical  providers,  public  health  officials,  health 
plan  sponsors  and  lastly  but  of  utmost  importance,  technical  advisors  who  brought 
scientific  rigor  to  the  process  of  defining  standard  performance  measures.  The  original 
purpose  of  HEDIS  was  to  provide  to  purchasers  and  consumers  of  healthcare  information 
based  on  outcomes  to  compare  managed  healthcare  plans  based  on  quality  and  not  just 
cost.  Eight  performance  measures  were  defined  by  the  Committee  on  Performance 
Measures,  one  of  which  is  "Member  Satisfaction,"  the  focus  of  this  paper.  The 
development  of  this  report  card  and  its  performance  measures  by  the  CPM  was  very 
important  in  standardizing  many  of  the  healthcare  industry's  views,  evaluations,  and 
basic  definitions.  The  million-dollar  question  has  always  been  "what  constitutes  quality 
healthcare?"  Through  HEDIS  and  the  efforts  of  the  CPM,  the  establishment  of  these 
performance  measures  has  addressed  this  question.  Additionally,  these  measures  have 
been  further  outlined  by  a  set  of  defined  categories  as  indicators.  (HEDIS  1997) 

The  construct  that  is  of  prime  importance  to  this  paper  is  the  definition  of  Patient 
Satisfaction  as  described  in  the  "Member  Satisfaction"  measurement.  This  contains 
seven  indicators  of  overall  satisfaction  which  are:  1)  Access:  the  ease  in  arranging  for  and 
getting  care;  2)    Communication:  the  level  of  explanation  and  attention  given  to  the 
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patient;  3)  Choice  and  Continuity:  the  ability  of  the  patient  to  see  the  doctor  of  your 
choice;  4)  Interpersonal  care:  the  extent  to  which  the  medical  staff  takes  a  personal 
interest  in  the  patient  and  his  or  her  medical  problems;  5)  Quality  and  Outcomes:  the 
overall  quality  of  care,  the  outcomes  of  medical  care,  and  how  much  the  patient  is  helped; 
6)  Technical  quality:  the  thoroughness  of  examinations  and  the  accuracy  of  the 
diagnosis;  and  7)  Finances:  the  level  of  protection  against  hardship  due  to  medical 
expenses  and  the  extent  of  arrangements  for  getting  help  with  financial  problems.  These 
seven  categories  have  been  defmed  as  indicators  of  overall  satisfaction  by  the  Committee 
on  Performance  Measures  and  are  used  as  a  working  definition  for  patient  satisfaction. 
(NCQA  1997) 
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IV.  METHODS 

A.  1996  DOD  ANNUAL  SURVEY  FORM  A 

The  Department  of  Defense  is  required  by  Congress  to  conduct  an  annual  survey 
of  its  active  duty  personnel,  retirees,  and  eligible  family  members  for  the  purpose  of 
measuring  accessibility,  usage,  and  satisfaction  in  a  range  of  health  care  services.  The 
first  of  these  surveys,  referred  to  as  the  1994-95  Health  Care  Survey  of  DoD 
Beneficiaries  (HCSDB)  was  completed  in  the  spring  of  1995.  This  paper  will  analyze 
portions  of  Form  A,  the  adult  survey  for  1996  HCSDB  conducted  by  OASD(HA)  Health 
Budgets  and  Programs.  A  point  of  contact  may  be  reached  at  (202)  761-7895,  Ext  246. 

1.  Participants 

A  sample  of  156,838  beneficiaries  was  selected  from  a  sampling  frame  of  6,455,915 
adult  sponsors  and  family  members  18  years  or  older  from  the  Defense  Enrollment 
Eligibility  Reporting  System  (DEERS)  of  eligible  beneficiaries  based  on  current 
eligibility  information  as  of  October  28,  1995.  The  sample  was  stratified  into  four 
beneficiary  subgroups:  1)  active  duty  personnel,  2)  family  members  of  active  duty 
personnel,  3)  retirees,  their  family  members,  and  survivors  under  age  65,  and  4)  retirees, 
their  family  members,  and  survivors,  age  65  and  over.  (WEST AT  1996) 

2.  Materials 

The  survey  is  a  nine-part  multi-faceted  questionnaire  that  covers  the  eight 
performance  measurements  outlined  by  the  HEDIS  requirements.  The  survey  consists  of 
84  questions  covering  the  following  sections:  I)  Your  Health  and  Daily  Activities;  II) 
Preventive  Health  Care  and  Health  Habits;  III)  Place  of  Medical  Care  and  Health 
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Insurance  Coverage;  IV)  Medical  Care  at  Military  Facilities;  V)  Medical  Care  at  Civilian 
Facilities;  VI)  Dental  Care;  VII)  TRICARE;  VDI)  Facts  About  You;  and  IX)  Comments. 
This  paper  will  primarily  look  at  section  IV,  Medical  Care  at  Military  Facilities.  Section 
IV  consists  of  questions  asking  if  the  respondents  had  received  any  healthcare  in  the  last 
12  months  from  a  Military  Treatment  Facility  (MTF).  If  the  answer  was  "yes"  then  they 
were  asked  about  the  following  issues:  1)  length  of  stay  in  the  hospital;  2)  number  of 
outpatient  visits;  3)  number  of  calls  to  obtain  an  appointment;  4)  length  of  waiting  time  in 
the  doctor's  office;  5)  length  of  time  to  travel  to  the  military  or  civilian  facility;  6) 
waiting  times  between  appointments  and  seeing  the  doctor  based  on  acuity  of  care;  7) 
overall  satisfaction;  and  8)  32  aspects  of  satisfaction  with  the  care  the  respondent 
received.  (WEST AT  1996) 

3.  Procedures 

The  survey  mailing  involved  four  waves  of  mailings  to  beneficiaries  between  April 
15,  1996  and  August  8,  1996.  The  multiple  mailings  were  scheduled  to  maximize 
response  rates  for  the  data  collection  effort.  First,  an  initial  notification  letter  was  mailed 
to  the  entire  sample.  Four  weeks  after  the  start  date  of  the  initial  notification  mailing,  a 
first  wave  of  surveys  was  mailed  to  the  entire  sample.  Next,  approximately  two  weeks 
after  the  first  wave  of  survey  mailings,  a  thank-you/reminder  letter  was  mailed.  Finally, 
four  weeks  after  the  mailing  of  the  follow-up  letters,  a  second  wave  mailing  of  surveys 
was  directed  to  those  who  had  failed  to  respond  to  previous  mailings.  The  field  was 
closed  for  targeted  mailings  on  September  10,  1996.  (WESTAT  1996) 

In  order  to  avoid  data  analysis  errors,  several  types  of  records  were  removed  from  the 
1996  HCSDB.    These  removals  were  performed  when:     1)  the  same  sample  member 
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filled  out  more  than  one  survey  (if  they  received  more  than  one  mailing  wave);  2) 
surveys  were  returned  from  sample  members  who  were  identified  as  deceased  or 
incarcerated;  3)  surveys  were  not  filled  out  by  the  intended  respondent;  4)  surveys  were 
blank  and  5)  there  was  a  gender  mismatch.  If  a  sample  member  returned  more  than  one 
survey,  then  the  first  wave  survey  was  accepted  and  placed  in  the  file.  If  the  first  wave 
survey  was  returned  blank,  then  the  second  wave  survey  was  selected.  (WEST AT  1996) 

A  final  step  was  taken  to  ensure  that  the  person  completing  and  returning  the  survey 
was  the  same  person  the  survey  was  mailed  to.  A  survey  record  was  judged  to  be  in  error 
if  the  gender  of  the  sample  member  was  different  than  the  self-reported  gender.  In  these 
cases,  it  was  evident  that  someone  other  than  the  intended  sample  member  completed  the 
survey.  Overall,  1,329  records  indicated  that  someone  other  than  the  sample  member 
completed  the  survey  were  removed.  (WEST AT  1996) 

Of  the  92,682  returned  surveys,  a  total  of  2,981  surveys  were  removed  from  the  1996 
HCSDB  data  file,  yielding  89,701  surveys.  The  overall  response  rate  was  58.18%.  The 
response  rates  for  the  subgroups  were:  45.0%  for  active  duty  ;  45.9%  for  family  members 
of  active  duty;  68.5%  for  retirees  under  65;  80.5%  for  retirees  age  65  and  older;  57.5%  for 
family  members  of  retirees  under  65  years;  and  72.8%  for  family  members  of  retirees  65 
and  older.  (WEST AT  1996) 
B.  ANALYSIS 

The  1996  Annual  Survey  is  a  complex  survey  that  uses  techniques  such  as  probability 
sampling,  stratification,  clustering  and  weighting.  The  purpose  in  analyzing  the  surveys  is 
to  identify  and  understand  the  causal  mechanisms  and  to  investigate  the  links    between 
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overall  patient  satisfaction  and  the  seven  factors  defined  by  the  HEDIS  report  card  along 
with  demographic  variables  such  as  sex,  age,  and  beneficiary  group. 

The  first  step  in  this  analysis  was  to  match  the  34  questions  from  the  patient  satisfaction 
section  of  the  Annual  Survey  to  the  NCQA  questions  found  in  the  HEDIS  Report  card. 
Tables  1  shows  the  comparison  of  each  of  the  DoD  Annual  survey  questions  with  the  report 
card  questions. 


Table  1.  Comparison  of  Questions 
HEDIS  Report  Card  Questions 


1996  DoD  Annual  Survey  Questions 


OVERALL 

The  overall  health  care  provided  through  your  health 
plan 

OVERALL  Q47a 

Q46a  I  am  satisfied  with  the  health  care  that  I 
received 

ACCESS:Arranging  for  and  Getting  Care 

a     Convenience  of  location  of  the  doctor's  office 

b.  Hours  when  the  doctor's  office  is  open 

c.  Access  to  specialty  care  if  you  need  it 
d     Access  to  hospital  care  if  you  need  it 

e.     Access  to  medical  care  in  an  emergency 
f      Arrangements  for  making  appointments  for 

medical  care  by  phone 
g.     Length  of  time  spent  waiting  at  an  office  to  see 

the  doctor 
h.     Length  of  time  you  wait  between  making  an 

appointment  for  routine  care  and  the  day  of  your 

visit 
i.      Availability  of  medical  information  or  advice  by 

phone 
j.     Access  to  medical  care  whenever  you  need  it 
k.     Services  available  for  getting  prescriptions  filled 

ACCESSrArranging  for  and  Getting  Care 
Q47a-ff 

a    Convenience  of  location  of  treatment 

b.  Convenience  of  hours 

d    Access  to  a  specialist  if  you  need  it 

e.  Access  to  hospital  care  if  you  need  it 

f .  Access  to  medical  care  in  an  emergency 

g.  Ease  of  making  appointments  for  health  care  by 
phone 

h.     Length  of  time  you  wait  at  office  to  see  the 

provider 
i.      Length  of  time  you  wait  between  making  an 

appointment  for  routine  care  and  the  day  of  your 

visit 
j .      Availability  of  health  care  information  or  advice 

by  phone 

c.  Access  to  health  care  whenever  you  need  it 

k.     Services  available  for  getting  prescriptions  filled 

TECHNICAL  QUALITY 

a     Thoroughness  of  examinations  and  accuracy  of 
diagnoses 

b.  Skill,  experience,  and  training  of  doctors 

c.  Thoroughness  of  treatment 

TECHNICAL  QUALITY  Q471-0 

1.      Thoroughness  of  examination 

m.    Ability  to  diagnose  your  health  care  problems 

n.     Skill  of  health  care  providers 

o.     Thoroughness  of  treatment 

QUALITY  AND  OUTCOMES 

a     The  outcomes  of  your  medical  care,  how  much 

you  are  helped 
b.     Overall  quality  of  care  and  service 

QUALITY  AND  OUTCOMES  Q47p-q 

p.     The  outcome  of  your  health  care  (how  much  you 

are  helped) 
q.    Overall  quality  of  health  care 

COMMUNICATION 

a     Explanations  of  medical  procedures  and  tests 

b.  Attention  given  to  what  you  have  to  say 

c.  Advice  you  get  about  ways  to  avoid  illness  and 
stay  healthy 

COMMUNICATION  Q47  r-u 

r.     Providers  explanation  of  health  care  procedures 
s.     Provider' s  explanation  of  medical  tests 
t      Attention  provider  gives  to  what  you  have  to  say 
u     Advice  provider  gives  you  about  ways  to  avoid 
illness  and  stay  healthy 
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Table  1.  Cont'd 

HEDIS  Report  Card  Questions 


1996  DoD  Annual  Survey  Questions 


INTERPERSONAL  CARE 

a     Friendliness  and  courtesy  shown  to  you  by 
doctors 

b.  personal  interest  in  you  and  your  medical 
problems 

c.  Respect  shown  to  you,  attention  to  your  privacy 
d     Reassurance  and  support  offered  to  you  by 

doctors  and  staff 

e.  Friendliness  and  courtesy  shown  to  you  by  staff 

f.  Amount  of  time  you  have  with  doctors  and  staff 
duringavisit 

INTERPERSONAL  CARE  Q47v-aa&dd 

w.    Courtesy  shown  to  you  by  health  care  providers 

x.     Provider's  concern  for  you  as  a  person 

dd.  Health  care  providers  personal  interest  in  the 

outcome  of  your  problem 
y.     Provider's  concern  for  your  privacy 
z.     Reassurance  and  support  offered  to  you  by  health 

care  providers 
v.     Courtesy  shown  to  you  by  administrative  staff 

(e.g.  receptionists) 
aa   Amount  of  time  with  health  care  providers  during 

a  visit 

CHOICE  AND  CONTINUITY 

a     Number  of  doctors  you  have  to  choose  from 

b.  Arrangements  for  choosing  a  personal  doctor 

c.  Ease  of  seeing  the  doctor  of  your  choice 

CHOICE  AND  CONTINUITY  Q47bb-cc 

bb.  Ability  to  choose  health  care  providers 
cc.   Ease  of  seeing  the  provider  of  your  choice 

FINANCES 

a     Protection  you  have  against  hardship  due  to 

medical  expenses 
b.     Arrangements  for  you  to  get  the  medical  care  you 

need  without  financial  problems 

FINANCES  Q47ee-ff 

ee.    Protection  you  have  against  financial  hardship 

due  to  medical  expenses 
ff     Help  with  arrangements  to  get  the  health  care 

you  need  without  financial  problems 

1.  Variable  Selection 

The  survey  data  consisted  of  89,701  observations  and  402  variables  that  included  all  of 
the  survey  questions  responses  and  the  constructed  and  scaled  variables  that  were  developed 
by  the  contractor,  WEST  AT,  to  support  analysis  of  the  data.  Section  IV  of  the  questionnaire 
deals  with  beneficiaries'  satisfaction  with  health  care  they  received  at  the  military  facilities. 
The  satisfaction  variables  used  in  the  analysis  from  the  questionnaire  are  from  questions  46a 
and  47a-ff  pertaining  to  the  military.  The  response  for  question  46a,  concerning  overall 
satisfaction,  had  a  Likert  scale  with  responses  of:  strongly  agree,  agree,  neither  agree  nor 
disagree,  disagree,  and  strongly  disagree.  These  were  assigned  values  in  order  from  5  to  1. 
The  remaining  questions  (Q47a-ff)  had  a  Likert  scale  with  responses  of:  excellent,  very 
good,  good,  fair,  poor  and  not  applicable.  The  individual  responses  for  each  of  these 
questions  were  given  a  score  from  5  down  to  1  plus  NA.    Then  the  set  of  questions  within 
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each  of  the  seven  HEDIS  factors  was  averaged  to  obtain  one  value  for  each  variable  that 
described  the  essence  of  the  outcome  being  measured.  So,  for  example,  the  responses  for 
the  four  questions  concerning  technical  quality  were  converted  to  numbers  on  the  1-5  scale; 
then  the  average  of  those  four  numbers  was  used  as  the  number  for  technical  for  that 
particular  respondent.  This  type  of  manipulation  of  the  data  is  acceptable  as  long  as  the 
original  scales  from  excellent  to  poor  are  not  referred  to  in  the  resulting  conclusions 
(Agresti  1984). 

2.  XPASTUSE 

From  Section  IV,  a  variable  called  XPASTUSE  was  constructed  by  the  contractor, 
WESTAT,  INC.,  to  represent  whether  a  beneficiary  had  used  any  military  care  (question  38) 
within  the  past  12  months.  The  XPASTUSE  variable  has  4  levels  of  use:  l=military  only, 
2=civilian  only,  3=both  military  and  civilian  and  ".' -NA,  no  use  of  any  medical  facility 
within  the  past  12  months.  All  records  with  XPASTUSE=1  (military  only)  were  extracted 
for  use  in  the  data  analysis. 

3.    Dependent  And  Independent  Variable  Selection 

The  dependent  variable  overall  was  the  response  variable  from  question  46a,  "I  am 
satisfied  with  the  health  care  that  I  received."  In  order  to  use  logistic  regression  and  to 
obtain  a  clear  distinction  between  dissatisfaction  and  satisfaction,  overall  was  converted  to  a 
dichotomous  variable  with  0=dissatisfied  and  l=satisfied.  Every  value  that  was  less  than  3 
was  set  equal  to  0  to  represent  dissatisfaction,  and  every  value  that  was  greater  than  or  equal 
to  3  was  set  to  1  to  represent  satisfaction.  For  the  trellis  graphic  display,  the  response 
variable  used  the  original  5  factor  levels  of  question  46a.  The  independent  explanatory 
variables  were  the  seven  HEDIS  averages  access,  technical,  quality,   communication, 
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interpersonal,    choice,   and  finances  and  three   demographic   variables   age,   sex  and 

beneficiary  group. 

C.    LOGISTIC  REGRESSION 

The  responses  of  survey  questions  are  considered  to  be  ordinal  categorical  data  where 
there  is  a  clear  ordering  of  categories,  but  the  absolute  distances  among  them  are  unknown 
(Agresti  1984).  However,  ordinal  responses  are  quantitative  in  the  sense  that  each  level  on 
its  scale  can  be  compared  in  terms  of  whether  it  corresponds  to  a  greater  or  smaller 
magnitude  of  each  characteristic  (Agresti  1984).  Alan  Agresti,  in  his  book  Analysis  of 
Ordinal  Categorical  Data  states  that  "The  distinction  regarding  whether  data  are  continuous 
or  discrete  is  often  less  important  than  whether  the  data  are  qualitative  (nominal)  or 
quantitative  (ordinal  or  interval)"  (1984).  Since  this  complex  survey  uses  ordinal  Likert 
scales  for  measurement  of  questions,  logistic  regression  is  an  appropriate  tool  for  data  when 
a  response  variable  exists  as  a  binary  categorical  variable,  and  the  explanatory  variables  are 
a  mix  of  categorical  and  quantitative  (Agresti  1984). 

1.    Data  Subsets  for  Analysis 

Three  approaches  were  taken  to  using  logistic  regression  in  analyzing  this  complex 
survey.  The  first  approach  took  a  random  sample  from  the  large  data  set  to  look  at  a  global 
view  of  patient  satisfaction  attitudes.  One  of  the  limitations  of  many  statistical  programs  is 
their  lack  of  ability  to  handle  missing  data.  A  large  portion  of  respondents  utilized  the  NA 
option  on  some  of  the  questions  47  a-ff,  and  SPLUS  4.5,  the  data  analysis  software,  omits  in 
logistic  regression  all  observations  with  any  NAs.  Next,  a  large  random  sample  of  25,000 
observations  was  taken  from  the  total  of  89,701  observations  without  replacement  to  form 
the  first  tier  of  the  data  set.    The  data  were  then  divided  by  the  XPASTUSE  variable  into 
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three  groups  according  to  whether  they  had  used  medical  treatment  facilities  in  a  military 
institution,  or  a  civilian  institution,  or  both.  Logistic  regression  was  used  on  the  data  set 
where  beneficiaries  had  only  used  military  healthcare. 

The  second  approach  was  to  first  sort  the  entire  data  sample  by  the  XPASTUSE  variable 
and  extract  all  of  the  beneficiaries  who  had  used  military  healthcare  in  the  past  year.  The 
data  were  then  sorted  by  region  to  look  and  see  if  there  were  any  regional  effects  of  patient 
satisfaction  among  beneficiary  groups.  Region  2,  the  Mid-Atlantic,  was  selected  from  the 
regions  to  be  used  as  a  representative  in  the  analysis.  Logistic  regression  was  then  used  on 
the  complete  subset  of  beneficiaries  who  had  used  only  military  healthcare  in  Region  2. 

The  third  approach  was  to  break  the  data  sets  down  even  furLier  within  Region  2,  by 
beneficiary  group,  to  determine  if  there  were  any  differences  among  the  variables  for 
patients  who  have  used  only  military  healthcare. 

2.    Procedures 

For  each  of  the  three  approaches,  a  data  frame  was  constructed  with  overall  as  the 
response  variable,  and  as  the  explanatory  variables,  the  seven  HEDIS  variables  and  the 
appropriate  demographic  variables.  In  addition  to  the  dependent  and  the  independent 
variables,  a  column  of  weights  developed  by  the  contractor,  WESTAT,  INC.,  was  used  in 
the  analysis  to  account  for  the  practical  limitations  of  the  survey  such  as  imprecise  survey 
frames  and  non-response  by  beneficiaries.  In  the  initial  models  the  response  variable  was 
modeled  by  the  explanatory  variables  appearing  only  as  the  main  effects.  Then  each  of 
these  was  used  in  a  stepwise  selection  process  that  permitted  the  addition  of  second  order 
interactions. 
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D.  TRELLIS  GRAPHICS 
1.  Explanation  of  Method 

Visualization  of  data  sets  is  one  of  the  most  important  techniques  in  statistical  analysis. 
The  Trellis  Graphics  Display  framework,  a  recently-developed  method  for  visualizing  data, 
is  based  on  a  system  of  panels  that  look  like  a  garden  trellis  (Becker  1998).  While  graphical 
visualization  techniques  have  been  available  at  least  since  the  1930s,  the  idea  of  Trellis  arose 
from  the  need  to  study  data  structures  with  complex  dependencies  and  interactions  among 
the  explanatory  variables  acting  upon  the  response  (Becker  1998).  Trellising  was  first 
described  in  William  S.  Cleveland's  book  Visualizing  Data,  written  in  1993,  which  used 
multi-panel  conditioning  for  up  to  two  conditioning  variables  (Becker  1998).  Univariate, 
bivariate,  and  trivariate  data  can  all  be  visualized  in  relation  to  a  physical  reality  of  space. 
But  for  any  data  set  that  has  four  or  more  dimensions,  called  a  hypervariate  data  set,  there  is 
no  way  to  visualize  all  of  the  variables  together  in  the  physical  space  (Cleveland  1993). 
Cleveland  found  that  the  solution  to  visualizing  hypervariate  data  is  to  visualize  it 
cognitively,  not  perceptually,  through  looking  into  the  subspaces  of  the  data  (Cleveland 
1993).  Since  most  common  graphical  approaches  for  visualizing  data  could  only  handle  two 
explanatory  variables,  the  need  for  visualization  of  three  or  more  explanatory  variables 
required  the  development  of  new  graphical  technology  (Becker  1996). 

From  1993  to  1996  William  Cleveland  and  Richard  Becker  developed  and  designed  a 
visualization  graphics  method  that  has  been  implemented  in  the  S-PLUS  graphics  to  view 
many  variables  on  a  multi-panel  display.  The  display  panels  are  laid  out  into  rows,  columns 
and  pages  on  which  subsets  of  data  are  graphed  and  displayed  using  well-known  graphical 
tools  such  as  scatterplots,  boxplots,  normal  quantile  plots,  and  3-D  plots.    Each  panel  shows 
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the  relationship  of  a  response  variable  to  one  or  two  independent  variables,  conditional  on 
the  values  of  some  additional  independent  variables.  (Becker  1996) 

The  Trellis  Display  is  a  three-way  rectangular  array  of  panels  with  columns,  rows,  and 
pages  that  are  read  starting  at  the  bottom  of  the  page  and  going  from  left  to  right  and  then  up 
the  page,  like  a  graph.  Figure  1  below  is  a  series  of  boxplots  that  has  a  response, 
explanatory  and  conditioning  variables. 


Boxplots  of  Overall  Satisfaction  by  Age,  Regions  and  Beneficiary  Groups 
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Figure  1.  Overall  Satisfaction  Modeled  by  Age. 


The  y  variable  is  the  response  variable  graphed  on  the  ^-axis  of  each  panel,  and  the  x 
variable  is  one  of  the  explanatory  variables,  graphed  on  the  x-axis.    In  this  example,  y  is 
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overall  satisfied  and  x  is  age.  The  conditioning  variables,  which  are  region  and  beneficiary 
group,  are  the  other  explanatory  variables  that  are  being  held  constant  at  a  particular  set  of 
factor  levels  or  for  continuous-data,  in  a  defined  interval.  The  names  of  the  conditioning 
variables  are  given  in  a  strip  label  at  the  top  of  each  panel.  Here,  the  first  conditioning 
variable  is  region  and  the  second  is  beneficiary  group.  Each  of  these  labels  has  a  dark 
shaded  bar  in  the  same  color  as  the  strip  label  of  the  panel  that  indicates  the  value  of  the 
variable  that  is  being  held  constant.  The  strip  label  and  the  bar  show  how  the  values  of  the 
conditioning  explanatory  variables  are  changing  as  they  go  throughout  the  trellis.  The 
colors  used  in  the  assembly  of  the  panels  are  analogous  colors  that  aid  in  the  decoding  of  the 
picture.  Insight  and  understanding  of  the  data  structure  is  found  by  examining  the 
relationships  between  the  response  and  the  conditioning  variables.  (Becker  1996) 

There  are  four  key  elements  which  make  the  Trellis  Graphics  Display  different  from 
other  graphics  methods.  The  first  is  the  "Main-Effects  Ordering."  This  display  method 
allows  the  ordering  of  the  response  variable  by  median,  mean,  or  any  special  ordering  that 
may  be  naturally  occurring  in  the  data.  At  the  same  time,  the  conditioning  variables  may 
also  be  ordered  to  obtain  a  natural  flow  of  data  that  can  easily  be  visualized.  Figure  2  shows 
a  graph  of  Region  2  with  the  HEDIS  variables  and  beneficiary  groups  in  alphabetical  order. 
(Becker  1996) 

It  is  difficult  to  read  this  graph  quickly  and  to  recognize  the  variables  which  are  rated 
highest  and  lowest  and  the  variables  for  which  the  sexes  differ. 
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Region  2  Satisfaction  Scores 
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Means  of  HEDIS  Measurements  for  Region  2 

Figure  2  Alphabetized  Factors. 

Figure  3  shows  how  the  main  effects  ordering  property  makes  the  visualization  of  the 
data  much  easier  to  read  and  to  quickly  see  the  differences  between  the  variables.  In  this 
figure,  the  HEDIS  variables  and  the  beneficiary  groups  are  reordered  from  lowest  median  to 
highest  median.  It  is  now  much  clearer  that  active  duty  beneficiaries  have  a  lower  median 
overall  than  the  other  groups  and  the  HEDIS  variable  choice  has  been  rated  distinctly  lower 
for  all  beneficiary  groups  and  overall  has  been  rated  distinctly  higher  by  each  beneficiary 
group 
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Region  2  Satisfaction  Scores 
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Means  of  HEDIS  Measurements  for  Region  2 
Figure  3  Main-effects  Ordering  by  Median. 

The  second  element  that  distinguishes  Trellis  from  other  graphics  methods  is  the  control 
of  the  "aspect  ratio."  The  aspect  ratio  the  height/width  ratio  of  the  data  rectangle,  that  is  the 
panel  rectangle  that  encloses  all  of  the  data.  Graphical  theory  has  shown  that  the  banking  of 
45°  gives  the  best  perception  of  relative  slopes  and  curvatures  of  a  line  (Becker  1996). 
Because  this  is  an  important  part  of  graphing,  the  Trellising  system  gives  the  users  direct 
access  to  define  the  aspect  ratio.  Figure  3  is  the  same  picture  as  Figure  2,  but  banked  by 
45°. 

The  third  element  that  makes  this  method  extremely  valuable  is  the  ability  to  condition 
on  more  than  two  variables.  The  ability  to  arrange  the  variables  allows  control  of  the 
explanatory  variables  while  viewing  multiple  comparisons  of  many  combinations  of 
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variables.  Figure  4  shows  the  histograms  of  "age"  while  the  regions  move  from  left  to 
right  and  the  beneficiary  groups  move  from  bottom  to  top.  This  ability  to  condition  on 
more  than  two  variables  allows  further  investigations  of  many  different  demographic 
subsets. 

Distribution  of  Age  by  Region  and  Beneficiary  Group 
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Figure  4.  Conditional  Variables. 

Lastly,  the  Trellis  software  allows  for  conditioning  variables  to  be  continuous.  It  has  a 
function  called  shingleQ  that  allows  the  user  to  divide  a  variable  into  overlapping  intervals, 
each  with  the  same  number  of  points,  to  use  continuous  variables  as  conditional  variables 
(Becker  1996).  This  is  extremely  important  for  uncovering  interactions  between 
explanatory  variables  since  it  allows  the  conditioning  variable  to  be  held  nearly  constant. 
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2.  Data  Subset 

Regions  1,  2,  9,  and  11  were  selected  from  the  fourteen  regions  based  on  their  location. 
These  included  the  three  Naval  teaching  hospitals,  The  National  Naval  Medical  Center  in 
Bethesda  MD,  Naval  Hospital  Portsmouth  in  VA,  Naval  Hospital  San  Diego  in  CA,  and  a 
Fleet  Hospital,  Naval  Hospital  Bremerton  in  WA.  These  regions  were  selected  to  use  in 
studying  the  interactions  among  variables  and  for  showing  the  HEDIS  variables  that  may 
help  military  medical  facilities  provide  better  care  for  their  patients. 

3.  Procedures 

In  order  to  visualize  the  relationships  among  variables  in  the  four  regions  these  steps 
were  taken.  The  first  step  was  to  see  an  overall  view  of  what  the  data  looked  like.  To  do 
this,  the  histogram  was  first  used  to  display  the  densities  of  the  different  ages  among 
beneficiary  groups  and  sex.  This  is  important  to  see  because  age  was  not  used  as  a 
conditioning  variable  since  beneficiary  groups  provide  decision  makers  with  a  better 
demographic  breakdown  based  on  the  levels  of  access  outlined  in  Title  10,  U.S. C.  Chapter 
55,  Sections  1071-1106  which  describes  the  entitlement  programs.  Next,  the  satisfaction 
scores  for  the  four  regions  were  shown  to  determine  if  there  are  any  areas  among  the  HEDIS 
factors  that  the  primary  military  facilities  in  each  region  can  use  to  provide  better  care  within 
each  subset  of  beneficiaries.  This  was  done  by  calculating  the  means  of  each  HEDIS  factor 
or  each  conditioning  variable  and  plotting  them  together  to  show  comparisons  and  to  see  an 
overview  of  the  data  for  each  of  the  four  regions. 

The  last  approach  investigated  the  complex  interactions  and  dependencies  among  the 
HEDIS  variables  and  the  demographic  variables.  Trellis  was  used  to  model  four  variables 
at  a  time  and  then  used  to  investigate  even  more  complex  interactions  with  five  variables. 
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This  step  used  boxplots  to  summarize  the  conditional  distributions.  In  a  boxplot,  the  dot 
represents  the  center  of  the  data  or  the  median  of  the  distribution.  The  upper  and  lower  ends 
of  the  box  represent  the  25th  and  75th  percentiles  and  therefore  the  distance  between  the  ends 
of  the  box  is  indicates  how  tightly  packed  the  values  are  around  the  median  (Cleveland 
1993). 
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V. 


RESULTS 


A.  LOGISTIC  REGRESSION 


The  first  approach  outlining  the  methods  was  to  take  a  random  sample  of  25,000  from  the 
data  set  of  89,701  responses.  After  subsetting  the  data  for  military  use  only  and  accounting 
for  missing  data,  the  model  used  a  sample  size  of  N=4,361  with  27  explanatory  variables. 
The  initial  main-effects  generalized  linear  model  that  included  regions  showed  all  of  the 
variables,  except  the  dummy  variables  for  regions  6  and  12  to  be  significant.  Table  2  shows 
the  results  of  this  approach. 

Table  2  Logistic  Regression  of  Main-effects  Model  with  Regions. 


Coefficient 

Value 

Std.  Error 

t  value 

(Intercept) 

-4.088 

0.034 

-117.206 

access 

0.628 

0.009 

64.550 

technical 

0.196 

0.011 

8.477 

quality 

0.800 

0.011 

71.953 

communication 

-0.145 

0.010 

-13.879 

interpersonal 

0.340 

0.010 

32.394 

choice 

0.321 

0.007 

45.566 

finances 

-0.147 

0.005 

-28.116 

Ben.Group2 

0.332 

0.015 

21.531 

Ben.Group3 

0.434 

0.019 

22.165 

Ben . Group4 

1.099 

0.059 

18.328 

Age 

0.009 

0.000 

14.650 

REGION2 

-0.619 

0.019 

-31.759 

REGION3 

0.264 

0.021 

12.511 

REGION4 

0.210 

0.026 

7.883 

REGION5 

0.326 

0.024 

13.042 

REGION6 

-0.014 

0.019 

-0.732 

REGION7 

0.068 

0.026 

2.543 

REGION8 

0.310 

0.022 

13.754 

REGION9 

0.757 

0.023 

32.362 

REGION10 

0.398 

0.036 

10.999 

REGION11 

0.407 

0.033 

12.130 

REGION12 

-0.043 

0.035 

-1.223 

REGION13 

0.186 

0.048 

3.830 

REGION14 

0.136 

0.019 

6.927 

sex 

-0.234 

0.012 

-18.079 

>nd 


One  of  the  troubling  results  found  in  the  main-effects  model  and  found  in  the  2    order 
interaction   model   was    a   negative   sign   for   the   coefficients   of  variables   such    as 
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communication,  and  finances.  The  negative  correlation  implies  that  the  patients  who  rated 
overall  satisfied  high,  were  less  satisfied  than  those  who  rated  overall  satisfied  low  for  those 
variables.  Similar  counter-intuitive  results  were  also  found  in  the  2nd  order  interaction 
models. 

The  above  models  were  very  complex  and  too  large  to  explain  with  clarity.  The  data  was 
broken  down  further  into  regions  because  of  the  scores  of  interactions.  Region  2  with  a 
sample  size  of  1351  was  selected  for  this  analysis.  The  results  for  the  main-effects  model 
and  the  2  order  interaction  model  were  similar,  showing  negative  values  for  the 
coefficients  of  the  main  variables  such  as  finances,  communication  and  interpersonal. 

As  a  final  step,  the  data  was  broken  down  by  beneficiary  group  to  see  if  there  were  any 
differences  among  the  types  of  beneficiaries  that  would  lead  to  interactions  and  negative 
coefficients.  Region  2  was  divided  into  beneficiary  groups  with  the  following  N:  active 
duty  N=493,  active  duty  family  members  N=268,  retirees  and  their  family  members  <65 
years  old,  N=390,  and  retirees  and  their  family  members  >=  65  years  old,  N=200.  The 
results,  using  stepwise  regression,  from  the  main-effects  model  by  beneficiary  group  showed 
variations  between  each  group.  In  Table  3,  the  active  duty  group  added  technical  as  a 
negative  coefficient  but  still  had  all  of  the  variables  as  significant. 

Table  3  Active  Duty  main-effects  model. 


Coefficient 

Value 

Std.  Error 

t  value 

(Intercept) 

-8.023 

0.067 

-119.455 

technical 

-0.651 

0.020 

-31.487 

access 

0.687 

0.016 

41.227 

finances 

0.419 

0.009 

-42.137 

choice 

0.654 

0.013 

48.826 

interpersonal 

0.407 

0.018 

21.984 

communication 

-0.312 

0.019 

-15.868 

quality- 

2.768 

0.023 

115.819 

sex 

0.364 

0.027 

13.288 

age 

0.031 

0.001 

26.781 
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In  Table  4,  the  active  duty  family  member  group  found  finances  and  access  to  be 
insignificant. 

Table  4  Active  Duty  Family  Members  main-effects  model 


Coefficient 

Value 

Std.  Error 

t  value 

(Intercept) 

-4.974 

0.091 

-54.264 

technical 

0.439 

0.036 

12.063 

access 

-0.009 

0.029 

-0.316 

finances 

0.003 

0.014 

0.270 

choice 

0.637 

0.020 

31.438 

interpersonal 

1.036 

0.035 

29.014 

communication 

-0.689 

0.027 

-24.811 

quality 

1.357 

0.035 

38.751 

sex 

-0.723 

0.046 

-15.682 

age 

-0.013 

0.001 

-7.774 

In  Table  5,  the  model  of  the  retirees  and  their  family  members  who  are  younger  than  age 
65  showed  all  of  the  variables  to  have  significance  except  interpersonal  and  did  not  have 
any  negative  correlation  for  the  variables  that  were  significant. 

Table  5  Retirees  and  Family  Members  <65  main-effects  model 


Coefficient 

Value 

Std.  Error 

t  value 

(Intercept) 

-8.001 

0.154 

-51.927 

technical 

0.989 

0.052 

19.019 

access 

0.756 

0.043 

17.527 

finances 

0.129 

0.024 

5.300 

choice 

0.475 

0.033 

14.283 

interpersonal 

-0.019 

0.046 

-0.421 

communication 

0.011 

0.048 

8.431 

quality 

0.564 

0.046 

12.260 

sex 

0.450 

0.045 

9.957 

age 

0.016 

0.001 

8.847 

And  lastly,  Table  6  gives  the  main-effects  model  for  the  retirees  and  their  family 
members  who  are  age  65  years  and  older.  It  showed  technical  and  access  as  non-significant 
and  had  negative  correlation  for  finances,  choice,  and  communication  with  the  response 
variable  overall. 


37 


Table  6  Retirees  and  Family  Members  >=  65  main-effects  model 


Coefficient 

Value 

Std.  Error 

t  value 

(Intercept) 

-15.982 

1.627 

-9.820 

technical 

0.193 

0.214 

0.901 

access 

0.103 

0.182 

0.568 

finances 

-0.275 

0.115 

-2.382 

choice 

-0.496 

0.268 

-2.356 

interpersonal 

3.439 

0.337 

10.198 

communication 

-0.918 

0.285 

-3.218 

quality 

2.245 

0.315 

7.116 

sex 

5.242 

0.411 

12.749 

age 

0.048 

0.021 

2.321 

The  2nd  order  models  by  beneficiary  groups  were  similar  to  the  first  two  approaches  with 
many  combinations  of  significant  interactions  between  explanatory  variables. 

The  three  approaches  above  used  logistic  regression  to  predict  overall  patient  satisfaction 
by  the  seven  HEDIS  variables  and  the  demographic  variables.  Each  primarily  showed  that 
almost  all  of  the  variables  are  important  with  some  variables  such  as  finance  and 
communication  had  some  negative  correlation  with  the  response  variable  overall.  In 
addition,  there  were  complicated  interactions  in  each  the  data  sets  among  the  demographic 
variables  and  the  HEDIS  measurements.  One  of  the  Trellis  Graphics  display  method's 
strength  is  its  ability  to  uncover  interactions  through  visualization  and  conditioning  of 
variables. 
B.  TRELLIS  GRAPHICS 

1 .    Age  Distributions  of  Regions  1, 2,  9,  and  11 

One  of  the  beneficial  features  of  Trellis  is  the  ability  to  view  distributions  of 
subgroups  at  a  glance.  The  analysis  of  the  patient  satisfaction  data  set  used  the  subgroups 
region,  beneficiary  group  and  sex.  Although  age  was  one  of  the  initial  demographic 
variables,  by  definition,  the  beneficiary  group  delineation  provided  an  idea  of  the  range 
of  ages  as  shown  in  Figure  5  for  female  respondents  and  Figure  6  for  male  respondents. 


38 


Distribution  of  Age  by  Region,  Beneficiary  Group,  and  Sex 
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Figure  5.  Histogram  of  Female  ages  by  Beneficiary  Group. 

In  row  1  of  Figure  5  there  is  only  a  small  number  of  female  active  duty  respondents  in 
each  region.  Figure  6  shows  a  comparison  of  the  distribution  of  ages  of  the  male 
respondents  for  the  given  regions.  Active  duty  males  had  the  largest  population  of 
respondents  in  each  of  the  regions;  in  each  case  the  smallest  group  was  male  family 
members  of  active  duty  personnel.  The  two  retiree  groups  had  approximately  the  same 
distribution  for  both  males  and  females  with  the  ages  of  the  younger  population  mostly 
falling  within  the  40-60  year  old  range  and  the  age  of  the  older  group  falling  within  the 
60-80  year  old  range. 
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The  number  of  returned  surveys  within  the  four  regions  are  different  with  Region  1, 
the  Northeast,  which  includes  the  National  Capital  area,  being  the  largest.  Region  1 1 ,  the 
Northwest,  has  the  smallest  response  rate  and  Region  2,  the  Mid-Atlantic,  or  Portsmouth, 
VA  area  and  Region  9,  Southern  California,  or  San  Diego  area  having  about  the  same 
distribution  of  sizes. 

Distribution  of  Age  by  Region,  Beneficiary  Group,  and  Sex 
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Figure  6  Histogram  of  Male  Ages  by  Beneficiary  Group. 

2.         Dotplots  of  HEDIS  Measurements 

In  order  to  help  military  medical  facilities  locate  key  issues  to  improve  healthcare  for 
their  patients,  the  different  scores  for  the  HEDIS  measurements  should  be  analyzed  for 
each  region.    The  dotplot  of  the  mean  scores  is  one  of  the  best  techniques  for  providing  a 
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highly  accurate  visual  decoding  of  data  (Becker  1996).  First,  the  mean  for  each  of  the 
HEDIS  variables  was  calculated  by  region,  beneficiary  group,  and  sex.  Then  Trellis 
graphics  was  used  to  plot  the  means  of  each  measurement  by  HEDIS  variable,  region, 
and  sex  in  an  easy-to-read  picture.  Figure  7  below  shows  all  four  regions  ordered  by  their 
overall  medians. 

Regions  1,2,9  and  11  Satisfaction  Scores 


= 


Overall  Satisfied 

Interpersonal 

Technical 

Quality 

Communication 

Access 

Finances 

Choice 


— R 


OverallSatjsfied 

Interpersonal 

Technical 

Quality 

Communication 

Access 

Finances 

Choice 


Rfiginn? 


2.0     2.5     3.0     3.5     40 

TIT     flH1T*|  RBtiri>PS«FM>=fldl 


©- 


FtirfPsUFNWiS 


Rfoinn? 


AriivfiniityFM 


Reoion2 


OverallSabslied 

Interpersonal 

Technical 

Quality 

Communication 

Access 

Finances 

Choice 


OverallSatisfied 

Interpersonal 

Technical 

Quality 

Communication 

Access 

Finances 

Choice 


Ropinn  1 1 


Rplirppitf 
Beoioa 


AcliyeDMtyFM 


iDntyF 
ionil 


Region  11 


-1 ' ' 3-1- 


Retlrpi>s«FM>=flg 


20     25     30     35     40 


RpginnO 


Anrh»riiityFM 


Reqionfl 


RBBr.e^iFM>-lK( 


Regionl 


ReaiuM 


etiveOutyFM 


Region! 


20      25      30      35      40 


2  0      2  5      3  0      3  5      4  0 


Female 
Male 


Means  of  HEDIS  Measurements 

Figure  7  Regions  1,  2,  9,  and  11. 

The  choice  variable  has  the  lowest  median  overall,  and  the  variable  overall  has  the 
highest  median.  Panel  (1,1)  in  the  lower  left  corner  shows  that  Region  2  has  the  lowest 
median  and  in  panel  (4,4),  retirees  and  their  family  members  over  65  years  old  has  the 
highest  median. 
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The  variable  choice  was  rated  the  lowest  by  each  beneficiary  group  in  each  of  the  four 
regions  with  the  exception  of  male  active  duty  family  members  rating  finance  lower  in 
Region  11.  It  is  also  interesting  to  note  that  the  women  in  Regions  1 1  and  1  are  very 
different  from  the  men.  There  seems  to  be  an  upward  trend  from  active  duty 
beneficiaries  to  retirees,  indicating  that  the  older  population  tends  to  be  happier  than  the 
younger  groups.  These  relationships  will  be  examined  in  greater  detail  as  each  of  the 
four  regions  is  discussed  individually. 

Grouping  the  regions  together  on  one  graph  provides  a  comparison  between 
regions.  However,  it  is  best  to  graph  them  separately;  that  way  the  greater  spread  of  the 
values  can  better  show  distinguishing  features  of  the  satisfaction  scores.  Region  2's 
satisfaction  scores  are  shown  in  Figure  8. 

The  variable  choice  here  is  shown  on  a  finer  scale  than  in  Figure  7  and  demonstrates  the 
difference  of  ratings  by  the  different  beneficiary  groups.  For  active  duty  members  (females: 
N=72,  males:  N=421)  and  their  families  (females :N=246,  males:  N=22),  women  gave 
choice  a  lower  score  than  the  men.  In  addition,  for  all  other  factors  except  finances  for  the 
active  duty  members  and  technical  for  the  family  members,  the  women  gave  lower  ratings 
than  the  men.  For  retirees  and  their  family  members  under  65  (females:  N=199,  males: 
N=191),  there  was  no  distinct  separation  between  the  sexes.  Finances  have  the  second- 
lowest  ratings  for  the  retiree  groups.  Overall,  the  older  beneficiaries  seemed  to  be  happier 
with  military  facilities  than  active  duty  members  and  their  families. 
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Region  2  Satisfaction  Scores 
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Figure  8  Region  2. 


Region  1 1,  shown  in  Figure  9,  had  the  second  lowest  overall  median  score.  Region  1 1 
was  very  different  from  the  other  regions  primarily  in  that  there  is  a  clear  distinction 
between  men  and  women  for  their  satisfaction  scores.  There  was  almost  a  full  point 
difference  between  males  and  females  in  almost  every  category  for  each  beneficiary  group. 
The  most  striking  difference  was  for  active  duty  family  members  who  rated  the  factor 
finances.  However,  on  closer  examination,  the  small  number  of  respondents  in  the  male 
family  member  group  (N=8)  may  somewhat  overstate  the  difference  compared  to  the  larger 
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group  of  females  (N=148). 


Region  1 1  Satisfaction  Scores 
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Figure  9  Region  1 1 . 

The  overall  satisfaction  score  for  the  retirees  and  their  family  members  age  65  years  and 
older  was  a  deviation  for  men  (N=55)  from  the  dominant  trend.  In  this  group,  overall  is 
rated  lower  than  the  next  four  highest  factors,  interpersonal,  technical,  quality, 
communication,  and  access.  If  overall  is  thought  to  be  approximately  an  average  of  the 
other  variables,  then  this  was  the  only  group  that  indicated  that  result.  With  the  exception  of 
this  one  value,  the  spread  from  the  beneficiaries'  ratings  from  all  other  variables  to  overall 
was  very  large  which  is  somewhat  an  anomaly.  The  last  noteworthy  finding  is  that  given  the 
low  number  of  active  duty  family  members  who  are  male,  females  across  all  of  the 
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beneficiary  groups  in  Region  11  were  almost  always  less  happy  than  males.  This  was 
shown  for  every  HEDIS  measurement  except  finance  for  the  active  duty  groups  and  for 
overall  satisfied  fox  retirees  and  family  members  younger  than  age  65. 

Region  9  had  the  second  highest  median  score  of  the  four  regions  as  shown  in  Figure  10. 
There  are  two  strikingly  different  findings  for  Region  9  that  were  distinct  from  those  in  the 
other  regions.  First,  as  shown  by  the  ordering  of  the  colors  in  the  key,  males  had  the  lower 
median  response  rate.  In  all  of  the  other  regions,  females  had  the  lower  median. 

The  second  finding  is  that  Region  9  was  the  only  region  where  active  duty  males 
(N=463)  rated  choice  lower  than  females  (N=67);  here  the  difference  in  the  spread  of  ratings 
is  approximately  a  full  point.  This  region  was  also  the  only  one  in  which  the  active  duty 
men  were  predominantly  less  happy  than  the  women.  This  is  also  seen  for  the  active  duty 
family  members  who  are  male:  though  there  are  only  14  men  in  this  group;  when  put  with 
the  463  active  duty  males  who  felt  similarly,  this  is  a  substantial  finding. 

Additionally,  the  overall  marks  were  also  very  high  in  this  region  which  was  not  what 
would  be  expected  with  such  a  score.  It  is  also  interesting  to  note  that  even  though  the 
males  in  the  two  active  duty  groups  were  less  happy  in  almost  all  of  the  HEDIS 
measurements,  they  still  rated  overall  (satisfied)  higher  than  the  females  in  Region  9. 
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Region  9  Satisfaction  Scores 
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Figure  10  Region  9. 


The  National  Capitol  Region  had  the  highest  median  among  the  four  regions.  Figure  1 1 
shows  the  satisfaction  scores  for  Region  1.  Within  this  region  the  variables  choice  and 
overall  stand  out  and  follow  a  trend  similar  to  that  in  the  other  regions.  However,  the  spread 
from  active  duty  members  (female:  N=128,  male:N=743)  to  retirees  and  their  family 
members  age  65  and  older  (females:  N=204,  males:  N=210)  for  choice  is  very  wide.  For 
retirees  and  their  family  members  younger  than  age  65,  the  males  (N=296)  were  more 
unhappy  than  the  females  (N=272)  for  the  measurements  finance  and  choice.  Also  the 
spread  between  the  sexes  in  all  of  the  beneficiary  groups  except  active  duty  members  is  very 
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noticeable  indicating  different  attitudes  and  feelings  among  men  and  women  respondents  in 
Region  1. 

Region  1  Satisfaction  Scores 
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Figure  11  Region  1. 

3.  Trellis  Interactions 

Interactions  among  variables  of  a  multivariate  data  set  occur  when  there  is 
interdependence  among  variables  at  different  levels  or  when  the  effect  of  a  variable  depends 
on  the  values  of  the  other  explanatory  variables.  The  logistic  regression  models  all  showed 
complex  interactions  occurring  among  the  HEDIS  variables,  between  the  HEDIS  variables 
and  the  demographic  variables,  and  between  the  demographic  variables  region,  sex  and 
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beneficiary  groups.  Interactions  can  cause  difficulties  in  using  logistic  regression  to  predict 
a  response  variable  when  they  are  too  numerous  to  explain  causality  clearly  and  concisely. 
If  a  data  set  has  explanatory  variables  that  have  strong  dependencies  among  themselves,  like 
the  HEDIS  measurements,  which  are  attitudes  of  respondents,  then  interactions  among 
variables  are  likely  and  can  be  difficult  to  identify.  The  Trellis  Graphics  framework  is 
extremely  helpful  in  seeing  interactions  and  viewing  complex  data  sets. 

Figure  12  is  a  graph  with  boxplots  of  overall  satisfaction  modeled  by  access  while 
holding  each  region  and  beneficiary  group  as  conditional  constants. 


Boxplots  of  Overall  Satisfaction  and  Access  by  Region  and  Beneficary  Group 
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Figure  12  Relationships  of  Variables  with  Few  Interactions. 
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This  figure  demonstrates  a  set  of  variables  that  have  few  or  no  interactions  among 
variables.  Interactions  are  seen  as  different  slopes  of  panels  as  the  eye  moves  through  the 
conditional  variables.  In  row  1  of  Figure  12,  as  the  eye  moves  from  left  to  right,  there  seems 
to  be  very  little  interaction  occurring  between  regions.  Each  region  has  approximately  the 
same  shape  and  slope  of  the  medians  (dots  within  the  boxplots)  and  the  sizes  of  the  boxes 
are  about  the  same,  indicating  similar  distributions  of  the  interquartile  ranges.  When 
looking  at  a  specific  region  such  as  Region  1,  panel  (1,1)  and  panel  (1,2)  appear  to  be 
approximately  the  same,  indicating  little  interaction  or  interdependency  within  Region  1 
among  beneficiaries  between  beneficiary  groups  in  the  first  two  groups  and  the  relationship 
between  access  and  overall  satisfaction.  Identifying  groups  of  variables  that  do  not  have 
interactions  may  help  the  decision-maker  understand  relationships  which  can  be  explained 
relatively  easily.  For  example,  as  access  increases,  the  rating  of  overall  satisfaction  by 
respondents  in  Region  1  who  are  active  duty  or  family  members  of  active  duty  will  increase 
by  approximately  the  same  amount.  In  other  words,  if  a  military  facility  would  like  to 
increase  overall  satisfaction,  then  there  is  consistent  relationship  between  that  variable  and 
access. 

In  the  1994-95  survey,  access  was  rated  as  the  lowest  measurement  indicating  it  was 
one  of  the  most  dissatisfying  aspects  of  care  for  beneficiaries.  For  the  1996  survey,  choice 
was  rated  the  lowest  and  access  moved  to  the  third-lowest  measurement,  pointing  to  some 
improvement.  The  finance  measurement  was  rated  as  the  second  lowest  measurement  on 
the  1996  survey.  The  measurement  consists  of  two  questions:  1)  "Protection  you  have 
against  financial  hardship  due  to  medical  expenses"  and  2)  "Help  with  arrangements  to  get 
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the  health  care  you  need  without  financial  problems."    Figure  13  is  the  graph  of  satisfied 
overall  modeled  by  finance  conditioning  on  regions  and  beneficiary  groups. 

Boxplots  of  Overall  Satisfaction  and  Finances  by  Region  and  Beneficiary  Group 
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Figure  13  Relationships  of  Variables  with  Interactions. 

This  picture  is  very  unusual  because  in  almost  every  panel  by  region  and  beneficiary 
groups,  the  finance  measurement  has  a  median  of  3.  The  response  by  active  duty  members, 
who  should  feel  that  the  military  will  cover  their  own  medical  expenses  was  expected  to  be 
higher  than  3.  Instead,  this  group  has  several  of  the  ratings  across  regions  that  were  lower 
than  the  median  value  of  3.  This  could  indicate  that  when  thinking  of  financial  security, 
these  active  duty  members  might  be  describing  their  security  as  it  relates  to  their  families, 
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not  as  individuals.  In  column  3,  which  is  Region  2,  there  are  some  interactions  occurring 
between  the  beneficiary  groups,  while  there  appear  to  be  only  minimal  interactions 
occurring  in  Region  1 1 . 

Figures  14  and  15  are  graphs  of  overall  satisfaction  modeled  by  the  choice  measurement 
by  region,  beneficiary  group  and  by  sex  (Figure  14:  female,  Figure  15:  male). 

Boxplots  of  Overall  Satisfaction  and  Choice  by  Region,  Beneficary  Group  and  Sex 
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Figure  14  Interactions  among  Choice  Variable  for  Females. 
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Boxplots  of  Overall  Satisfaction  and  Choice  by  Region,  Beneficary  Group  and  Sex 
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Figure  1 5  Interactions  among  Choice  Variable  for  Men. 

The  choice  variable  had  the  lowest  mean  satisfaction  score  of  all  of  the  HEDIS 
measurements.  The  variable  is  defined  by  the  questions:  1)  "Ability  to  choose  health  care 
providers"  and  2)  "Ease  of  seeing  the  provider  of  your  choice."  This  variable  relates  to 
what  are  really  two  separate  issues,  choice  of  provider  and  continuity  of  care.  There  are 
many  interactions  going  on  between  beneficiary  groups  as  seen,  for  example,  by  starting  at 
the  bottom  of  each  column  and  going  up,  holding  the  region  constant.  This  is  also  true 
among  the  different  beneficiary  groups  across  regions.  In  Figure  14,  there  are  very  few 
females  who  rate  choice  above  the  value  3.  The  active  duty  females  seem  to  be  very 
unhappy  in  Regions  2  and  1 1  and  only  slightly  happier  in  Regions  1  and  9. 
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4.  Complex  5-Way  Interactions 

Figures  16  through  19  are  a  set  of  4  pages  for  a  5-way  interaction  with  overall  satisfied 
as  the  response  variable.  Choice  is  the  continuous  explanatory  variable  and  the  conditional 
variables  are  communication,  region  and  beneficiary  group.  Each  of  the  four  figures 
represents  a  beneficiary  group  beginning  with  active  duty  members.  .  Below  is  Figure  16, 
which  shows  the  active  duty  members  for  each  region. 

Overall  Satisfaction  and  Choice  by  Communication,  Region,  and  Beneficiary  Group 
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Figure  16  5-Way  Interactions  for  Active  Duty  Members. 

Each  row  on  each  page  shows  how  choice  is  modeled  by  overall  satisfaction  holding 
communication  constant  for  one-point  intervals.  Each  column  on  each  page  is  a  comparison 
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of  regions  for  each  level  of  communication  while  choice  and  overall  satisfaction  are  the 
response  and  explanatory  variables.  The  level  of  interaction  on  these  four  figures  are  very 
complicated  and  difficult  to  interpret;  however,  Trellis  provides  the  ability  to  see 
movements  within  the  conditional  variables  which  may  lead  to  insights  of  multi-level 
interactions  among  variables. 

In  Figure  16,  columns  1  and  2  depict  the  first  two  levels  of  communication  as  is  shown 
by  the  shaded  portion  of  the  strip  label.  As  each  level  of  communication  progresses  up  one 
interval,  interactions  are  occurring  within  each  region.  Figure  17  shows  the  interactions  for 
active  duty  family  members.  There  appear  to  be  interactions  for  all  levels  of  communication 
except  the  4  to  5  intervals  for  communication  in  column  4.  The  slopes  seem  to  be 
approximately  the  same  although  there  are  overall  satisfaction  levels  that  do  not  have  any 
observations  such  as  in  Region  1 1  and  Region  9.  Active  duty  family  members'  responses 
also  appear  to  have  rated  choice  lower  than  the  corresponding  levels  of  communication 
which  is  shown  by  the  ratings  in  almost  all  of  the  panels  being  to  the  left  of  the  shaded  bar 
that  indicates  what  level  of  communication  is  being  measured.  This  shows  the  same  results 
as  the  dotplots  which  is  to  show  that  choice  is  rated  lower  than  communication. 
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Figure  17  5-Way  Interactions  for  Active  Duty  Family  Members. 

Next,  Figure  18  of  retirees  and  their  family  members  less  than  65  year  old,  shows 
interactions  throughout  the  page,  with  one  possible  exception  of  the  slopes  of  the 
communication  interval  from  2  to  3,  which  look  similar  in  column  3. 
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Figure  18  5-Way  Interactions  for  Retirees  and  Family  Members  <  65. 

The  final  beneficiary  group  in  this  example  showing  multi-panel  interactions  is  the 
retirees  and  their  family  members  who  are  age  65  years  and  older.  This  group  had  the 
highest  median  of  overall  satisfaction,  reflected  by  the  small  number  of  observations  in 
column  1  of  Figure  19.  But  even  in  this  group  there  are  still  the  ratings  of  overall 
satisfaction  given  choice  that  are  lower  than  the  defined  interval  for  communication. 

These  four  graphs  of  5-way  interactions  are  difficult  to  interpret  unless  there  is  some 
expert  who  is  knowledgeable  about  the  specific  beneficiary  groups  within  each  region.  In 
this  type  of  situation,  the  complex  interactions  could  provide  more  insight  into  the  attitudes 
that  are  expressed  through  the  survey. 
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Figure  19  5-Way  Interactions  for  Retirees  and  Family  Members  >=  65. 

5.  Overview  of  all  14  Regions 

Figures  20  and  21  provide  a  comparison  of  the  satisfaction  scores  for  all  14  regions. 
In  Figure  20,  the  two  groups  that  have  the  lowest  median  are  active  duty  members  and 
active  duty  family  members.  The  region  having  the  lowest  median  is  Region  3  and  the 
region  that  has  the  highest  median  is  Region  5.  The  general  trend  of  women  being  less 
satisfied  with  healthcare  is  carried  through  most  of  the  regions.  However,  there  is  one 
exception  to  this  for  one  of  the  beneficiary  groups  in  Region  5.  Active  duty  males 
appear  to  be  noticeably  less  happy  than  the  females  as  shown  in  panel  (7,2).    Choice 
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remains  as  the  lowest  measurement  across  all  regions  and  active  duty  females  in  Region 
12  are  shown  in  panel  (5,1)  as  being  the  most  dissatisfied. 

Regions  1:14  Satisfaction  Scores 


Female 
Male 


Means  of  HEDIS  Measurements 
Figure  20  Regions  1:14  Active  Duty  and  Their  Family  Members  Groups. 

Figure  21  shows  that  the  beneficiary  groups  that  have  the  highest  median  for  all  of  the 
regions  are  the  two  retiree  groups.  Here  the  same  general  trends  are  seen  as  before  with 
the  measurements  choice  and  overall  satisfied.  Retirees  across  all  regions  are  much  more 
satisfied  with  healthcare  than  active  duty  members  and  their  families. 

The  explanation  for  this  occurrence  may  be  due  to  the  self-selection  of  retirees  and 
their  family  members  in  wanting  to  have  access  to  the  Military  Treatment  Facilities. 
These  two  groups  of  beneficiaries  have  the  least  availability  and  access  to  care;  when 
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they  do  gain  access,  they  are  likely  to  be  extremely  satisfied  with  that  episode  of  care.  In 
contrast,  active  duty  members  are  directed  to  go  primarily  to  the  Military  Treatment 
Facilities  and  do  not  generally  have  a  choice  in  the  location  of  where  they  may  receive 
medical  care.  Active  duty  family  members  do  have  a  choice  in  selecting  the  location  of 
care,  but  may  be  constrained  by  financial  concerns  in  those  regions  that  did  not  have 
TRICARE,  or  may  be  unhappy  with  their  situation  in  those  regions  that  did  have 
TRICARE 

Regions  1:14  Satisfaction  Scores 
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Figure  21  Regions  1:14  for  Retirees  and  their  Family  Members 
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VI.  CONCLUSIONS 
A.  DISCUSSION 

The  primary  objective  of  this  paper  was  to  determine  if  any  of  the  HEDIS 
measurements  would  indicate  areas  among  patient  satisfaction  scores  that  could  be  used 
by  decision-makers  to  improve  patient  satisfaction.  In  addition  to  understanding  these 
seven  areas,  another  important  question  was  whether  there  are  differences  in  satisfaction 
between  men  and  women  or  between  the  four  beneficiary  groups. 

First  logistic  regression  was  used  to  model  overall  satisfaction  by  the  seven  HEDIS 
variables  and  by  the  demographic  variables  used  to  describe  the  respondent  population. 
Although  this  technique  provided  insight  into  the  complexities  of  the  data,  it  raised  more 
questions  than  answers.  One  could  just  conclude  that  all  of  the  variables  and  interactions 
among  variables  are  significant.  The  problem  with  this  approach  is  that  just  determining 
that  all  of  the  variables  are  important,  indicated  by  the  largest  coefficients,  does  not 
describe  the  data  properly  or  explain  the  negative  coefficients  for  the  main-effects 
models.  The  relationships  among  the  variables  were  strongly  interdependent  and  to 
understand  them  and  how  to  improve  patient  satisfaction,  that  assumption  is  not  enough, 
so  an  additional  analytical  approach  was  needed. 

The  Trellis  Graphic  framework  provided  an  excellent  tool  to  visualize  and  understand 
the  complex  relationships  between  variables  and  the  interactions  of  variables.  One  of  the 
most  important  findings  is  that  overall  satisfaction  is  not  an  accumulation  of  the  seven 
HEDIS  factors  because  the  overall  satisfaction  measurement  was  rated  consistently 
higher  than  any  other  variable.  It  was  expected  that  overall  satisfaction  would  be 
approximately  an  average  of  the  variables  that  define  it.  A  possible  explanation  could  be 
related  to  the  positioning  of  the  question.    The  question  concerning  overall  satisfaction 

61 


with  the  health  care  appeared  in  the  survey  before  the  specific  questions  concerning 
access,  technical,  quality,  choice  and  continuity,  communication,  finances,  and 
interpersonal  skills.  There  seems  to  be  a  design  effect  created  by  the  order  of  the 
questions  on  the  survey  that  leads  respondents  to  answer  to  this  early  question  more 
positively  than  to  the  other  questions  that  make  up  the  HEDIS  variables.  If  a  decision- 
maker were  to  use  only  this  score  as  an  indicator  of  patient  satisfaction,  his  or  her 
audience  could  potentially  be  mislead. 

The  results  of  the  Trellis  graphics  for  the  four  defined  regions  suggests  that  across 
beneficiary  groups  the  measurement  that  can  have  the  greatest  impact  on  improving 
patient  satisfaction  is  choice,  which  includes  continuity  of  care.  In  a  study  conducted  for 
Kaiser  Permanente  of  Northern  California,  it  was  found  that  choice  in  choosing  one's 
own  primary  care  manager  was  the  single  most  important  factor  in  overall  satisfaction 
(Greene  1998).  This  analysis  would  indicate  military  beneficiaries  are  not  much  different 
from  their  counterparts  in  the  civilian  health  plans  in  regard  to  wanting  to  choose  their 
own  doctor  and  have  continuity  of  care.  Active  duty  members  rated  choice  in  the  lowest 
ranges  of  the  scale,  indicating  their  extreme  unhappiness  with  their  ability  to  choose  their 
provider.  In  the  past,  there  has  been  anecdotal  evidence  to  suggest  that  there  is  a  feeling 
among  active  duty  members  that  they  have  less  choice  than  dependents  and  retirees  do. 
This  analysis  clearly  indicates  this  perception  by  the  active  duty  population  through  their 
low  ratings  of  choice  in  every  region. 

One  answer  to  this  problem  might  be  the  TRICARE  program.  TRICARE  was 
developed  as  a  combination  of  the  strong  points  of  the  civilian  sectors'  managed  care 
plans  to  provide  patients  with  better  access,   choice  and  quality  of  care.      Under 
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TRICARE,  active  duty  members  are  automatically  enrolled  in  TRICARE  Prime,  which 
means  they  are  mostly  enrolled  at  the  military  treatment  facilities.  Regions  9  and  1 1  had 
substantial  experience  with  TRICARE  at  the  time  this  survey  was  conducted  but  Regions 
1  and  2  had  not  begun  the  TRICARE  program.  There  did  not  appear  to  be  a  consistent 
difference  in  ratings  between  regions  that  had  TRICARE  and  those  that  did  not,  possibly 
indicating  that  TRICARE  does  not  have  an  effect  on  how  patients  are  managed  within  the 
military  treatment  facilities.  However,  further  studies  would  be  needed  to  examine  the 
degree  of  interaction  between  these  variables. 

In  response  to  the  question  of  whether  there  are  differences  in  satisfaction  between 
men  and  women,  the  answer  is  yes.  There  are  some  exceptions  such  as  in  Region  9, 
where  active  duty  males  were  less  satisfied  in  almost  every  measurement  than  females, 
but  the  primary  trend  was  that  women  are  less  happy  than  men  across  beneficiary  groups 
and  regions.  This  was  particularly  true  for  Regions  1  and  11.  Research  done  by 
Gregory  Pascoe  (1983)  indicated  that  in  general  females  tend  to  be  more  happy  than  men, 
which  is  opposite  of  the  findings  for  these  four  regions  for  military  treatment  facilities. 

Studies  have  shown  that  older  patients  tend  to  be  happier  than  younger  patients,  and 
this  analysis  agrees  with  those  findings.  The  oldest  beneficiary  group,  retirees  age  65  and 
older  was  the  most  satisfied  group  across  all  regions.  This  was  not  what  was  initially 
expected  since  this  group  of  beneficiaries  fall  under  Medicare  and  have  the  lowest  level 
of  accessibility  to  military  treatment  facilities.  However,  since  they  have  been  able  to 
gain  access,  they  seem  to  be  extremely  happy  with  the  care  they  have  received,  possibly 
because  the  healthcare  is  perceived  as  high  quality  while  requiring  little  or  no  out-of- 
pocket  expenses. 
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On  a  final  note,  this  analysis  and  the  resulting  conclusions  would  not  have  been 
possible  without  the  use  of  the  Trellis  Graphics  framework.  This  graphing  technique 
provided  a  new  way  to  look  into  hypervariate  data  and  break  the  barrier  of  graphing  for 
up  to  three  dimensions.  This  would  have  been  impossible  only  a  few  years  ago.  This 
technique  will  revolutionize  the  way  multivariate  data  sets  can  be  used  when  other 
statistical  techniques  are  limited  by  interdependencies  among  variables.  Trellis  Graphics 
can  be  especially  useful  in  understanding  complex  behavioral  surveys,  like  the  DOD 
Annual  Survey,  that  typically  have  discrete  ordinal  scales.  Trellis's  strength  lies  in  its 
seemingly  simple,  easy  to  read  graphs  that  make  the  analysis  of  complex  data 
understandable.  This  method  was  extremely  useful  in  comparing  multiple  demographics 
of  subgroups  of  beneficiaries  that  enabled  additional  insight  into  trends  among  HEDIS 
measurements  for  age  and  sex.  Real-life  data  measuring  attitudes  and  feelings  often  have 
complexities  and  interdependence  that  restrict  the  use  of  statistical  tools  such  as  logistic 
regression.  The  Trellis  graphics  allowed  the  visualization  of  the  interactions  that  the 
logistic  regression  models  were  capturing  but  not  explaining. 
B.  RECOMMENDATIONS 

The  military  treatment  facilities  are  the  primary  location  of  care  for  active  duty 
members,  yet  this  beneficiary  group  was  the  least  satisfied  for  almost  every  HEDIS 
measurement.  This  indicates  that  there  is  a  perception  among  active  duty  members,  not 
held  by  the  retiree  groups,  that  health  care  in  the  military  treatment  facilities  is 
inadequate.  Since  the  active  duty  group  is  the  primary  customer  of  military  treatment 
facilities,  it  is  recommended  that  further  studies  be  conducted  to  pinpoint  the  reasons  for 
these  perceptions. 
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The  second  recommendation  is  to  include  a  question  on  overall  satisfaction  at  the  end 
of  the  32  HEDIS  measurement  questions  identical  to  the  one  at  the  beginning.  This  may 
allow  for  a  different  measurement  of  overall  satisfaction  and  would  also  measure  the 
effect  of  asking  the  question  twice  to  see  if  there  is  a  large  variation  in  responses  caused 
by  the  intervening  HEDIS  measurement  questions. 

The  third  recommendation  is  the  delete  the  NA  options  on  the  32  questions  on  patient 
satisfaction.  By  providing  that  option,  analysis  of  the  questions  is  affected  because  many 
statistical  tools  handle  them  by  deleting  each  response  with  any  NAs.  This  has  the  effect 
of  decreasing  the  number  of  responses  by  up  to  approximately  50%. 

The  final  recommendation  is  to  explore  the  use  of  Trellis  Graphics  for  individual 
catchment  areas.  Valuable  information  can  be  obtained  from  seeing  and  understanding 
the  responses  of  patients  who  have  used  a  specific  hospital;  this  would  provide 
information  and  understanding  to  decision-makers  at  the  lower  levels. 
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